Electronic device and method for generating image data

ABSTRACT

A method of an electronic device including an image sensor that acquires an optical signal corresponding to an object and a controller that controls the image sensor, is provided. The method includes identifying a mode for generating an image corresponding to the object by using the optical signal, determining a setting of at least one image attribute to be used for generating the image at least based on the mode, generating image data by using pixel data corresponding to the optical signal at least based on the setting, and displaying the image corresponding to the object through a display functionally connected to the electronic device at least based on the image data.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 16/235,357, filed on Dec. 28, 2018, which application is acontinuation application of prior application Ser. No. 15/286,083, filedon Oct. 5, 2016, which has issued as U.S. Pat. No. 10,200,646 on Feb. 5,2019 and was based on and claimed priority under 35 U.S.C § 119(a) of aKorean patent application number 10-2015-0139712, filed on Oct. 5, 2015,in the Korean Intellectual Property Office, the disclosure of which isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The present disclosure relates to an apparatus and a method forgenerating image data in an electronic device. More particularly,aspects of the present disclosure are related to an apparatus and methodfor generating image data by an image sensor having an image pixelarray.

BACKGROUND

The number of various services and additional functions provided by anelectronic device are gradually increasing. In order to increase theeffective value of the electronic device and meet various demands ofusers, various applications that are executable by the electronic devicehave been developed.

The electronic device may photograph an object through a camerainstalled in the electronic device, and the camera may include an imagesensor that detects the object. The electronic device may detect anobject or an image through, for example, the image sensor. The imagesensor may be configured, for example, in the unit of a plurality ofpixels, and each pixel unit may consist of, for example, a plurality ofsubpixels. The image sensor may include an array of small photodiodescalled, for example, pixels or photosites. For example, the pixel doesnot extract a color from light and may convert a photon of a widespectrum band into an electron. Accordingly, the pixel of the imagesensor may receive, for example, only the light of a band required foracquiring the color among lights of the wide spectrum band.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide a method of generating image data by an imagesensor having an image pixel array including a plurality of subpixel anda method of generating information for calculating a phase difference.An image sensor functionally connected to an electronic device mayoutput signals of all subpixels and then separate phase differenceinformation and image data, but may not effectively provide a maximumoutput of the image sensor. For example, processing signals output fromall subpixels may increase power consumption.

In accordance with an aspect of the present disclosure, a method of anelectronic device including an image sensor that acquires an opticalsignal corresponding to an object and a controller that controls theimage sensor is provided. The method includes identifying a mode forgenerating an image corresponding to the object by using the opticalsignal, determining a setting of at least one image attribute to be usedfor generating the image at least based on the mode, generating imagedata by using pixel data corresponding to the optical signal at leastbased on the setting, and displaying the image corresponding to theobject through a display functionally connected to the electronic deviceat least based on the image data.

In accordance with another aspect of the present disclosure, anelectronic device is provided. The electronic device includes a display,an image sensor configured to acquire an optical signal corresponding toan object, and at least one processor. The at least one processor isconfigured to identify a mode for generating an image corresponding tothe object by using the optical signal, determine a setting of at leastone image attribute to be used for generating the image at least basedon the mode, generate image data by using pixel data corresponding tothe optical signal at least based on the setting, and display the imagecorresponding to the object through a display functionally connected tothe electronic device at least based on the image data.

In accordance with another aspect of the present disclosure, anelectronic device is provided. The electronic device includes an imagesensor configured to acquire an optical signal corresponding to anobject, the image sensor including a plurality of unit pixels, at leastone of the plurality of unit pixels including a first subpixel and asecond pixel, and at least one processor functionally connected to theimage sensor. The at least one processor is configured to acquire pixeldata corresponding to the object at least based on optical signal byusing the image sensor, identify a setting of at least one imageattribute to be used for generating an image corresponding to theobject, determine a phase difference of the image by using a firstsignal corresponding to the first subpixel and a second signalcorresponding to the second subpixel when the setting meetspredetermined conditions, and refrain from determining the phasedifference when the setting does not meet the predetermined conditions.

An electronic device and a method according to various embodiments ofthe present disclosure may output, for example, image data in the unitof subpixels or pixels according to a use mode of the user, therebyeffectively using an image sensor. Further, an electronic device havingan image sensor including at least two subpixels (for example,photodiodes) may output image data at a high frame rate or with lowpower, so that the user can use information and/or perform a functionthat the user desires and to improve the usability of the electronicdevice.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 illustrates an electronic device within a network environment100, according to an embodiment of the present disclosure;

FIG. 2 is a block diagram of an electronic device according to anembodiment of the present disclosure;

FIG. 3 is a block diagram of a program module according to an embodimentof the present disclosure;

FIG. 4 illustrates an example of an electronic device that outputs imagedata according to an embodiment of the present disclosure;

FIG. 5 illustrates an example of a configuration of a pixel array of animage sensor according to an embodiment of the present disclosure;

FIG. 6A schematically illustrates a unit pixel having two photodiodes inan image sensor according to an embodiment of the present disclosure;

FIG. 6B schematically illustrates a unit pixel having four photodiodesin an image sensor according to an embodiment of the present disclosure;

FIG. 6C illustrates a structure of a unit pixel having two photodiodesaccording to an embodiment of the present disclosure;

FIG. 6D is a circuit diagram of a unit pixel having two photodiodes inan image sensor according to an embodiment of the present disclosure;

FIG. 6E illustrates an example of a time chart of each switch in a unitpixel according to an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating an operation for generating imagedata according to an embodiment of the present disclosure;

FIG. 8A illustrates an example in which a unit pixel of an image sensorincludes two subpixels according to an embodiment of the presentdisclosure;

FIG. 8B illustrates an example in which a unit pixel of an image sensorincludes four subpixels according to an embodiment of the presentdisclosure;

FIG. 9 illustrates an example of a process for outputting image data ina first mode according to an embodiment of the present disclosure;

FIG. 10 illustrates an example of a process for outputting image data ina second mode according to an embodiment of the present disclosure;

FIG. 11A illustrates an example of a process for outputting image datain a third mode according to an embodiment of the present disclosure;

FIG. 11B illustrates an example of a process for outputting image datain a fourth mode according to an embodiment of the present disclosure;

FIG. 12 illustrates an example of a process for selectively combiningsignals of a subpixel level in the fourth mode according to anembodiment of the present disclosure;

FIG. 13A illustrates an example of a channel for transmitting generateddata and/or information for calculating a phase difference according toan embodiment of the present disclosure;

FIG. 13B illustrates an example for transmitting image data andinformation for calculating a phase difference through a predeterminedchannel according to an embodiment of the present disclosure;

FIG. 13C illustrates an example for transmitting image data andinformation for calculating a phase difference through a predeterminedchannel according to an embodiment of the present disclosure;

FIG. 14 is a flowchart illustrating an operation for generating an imageaccording to an embodiment of the present disclosure; and

FIG. 15 is a flowchart illustrating an operation for generating imagedata according to another embodiment of the present disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the present disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding, but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thepresent disclosure. In addition, descriptions of well-known functionsand constructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but are merely used by theinventor to enable a clear and consistent understanding of the presentdisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of the presentdisclosure is provided for illustration purposes only and not for thepurpose of limiting the present disclosure as defined by the appendedclaims and their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

As used herein, the expression “have”, “may have”, “include”, or “mayinclude” refers to the existence of a corresponding feature (e.g.,numeral, function, operation, or constituent element such as component),and does not exclude one or more additional features.

In the present disclosure, the expression “A or B”, “at least one of Aor/and B”, or “one or more of A or/and B” may include all possiblecombinations of the items listed. For example, the expression “A or B”,“at least one of A and B”, or “at least one of A or B” refers to all of(1) including at least one A, (2) including at least one B, or (3)including all of at least one A and at least one B.

The expression “a first”, “a second”, “the first”, or “the second” usedin various embodiments of the present disclosure may modify variouscomponents regardless of the order and/or the importance but does notlimit the corresponding components. For example, a first user device anda second user device indicate different user devices although both ofthem are user devices. Similarly, a first element may be termed a secondelement, and a second element may be termed a first element withoutdeparting from the scope of the present disclosure.

It should be understood that when an element (e.g., first element) isreferred to as being (operatively or communicatively) “connected,” or“coupled,” to another element (e.g., second element), it may be directlyconnected or coupled directly to the other element or any other element(e.g., third element) may be interposer between them. In contrast, itmay be understood that when an element (e.g., first element) is referredto as being “directly connected,” or “directly coupled” to anotherelement (second element), no elements (e.g., third element) areinterposed between them.

The expression “configured to” used in the present disclosure may beexchanged with, for example, “suitable for”, “having the capacity to”,“designed to”, “adapted to”, “made to”, or “capable of” according to thesituation. The term “configured to” may not necessarily imply“specifically designed to” in hardware. Alternatively, in somesituations, the expression “device configured to” may mean that thedevice, together with other devices or components, “is able to”. Forexample, the phrase “processor adapted (or configured) to perform A, B,and C” may mean a dedicated processor (e.g., embedded processor) onlyfor performing the corresponding operations or a generic-purposeprocessor (e.g., central processing unit (CPU) or application processor(AP)) that can perform the corresponding operations by executing one ormore software programs stored in a memory device.

The terms used herein are merely for the purpose of describingparticular embodiments and are not intended to limit the scope of otherembodiments. A singular expression may include a plural expressionunless they are definitely different in a context. Unless definedotherwise, all terms used herein, including technical and scientificterms, have the same meaning as those commonly understood by a personskilled in the art to which the present disclosure pertains. Such termsas those defined in a generally used dictionary may be interpreted tohave the meanings equal to the contextual meanings in the relevant fieldof art, and are not to be interpreted to have ideal or excessivelyformal meanings unless clearly defined in the present disclosure. Insome cases, even the term defined in the present disclosure should notbe interpreted to exclude embodiments of the present disclosure.

An electronic device according to various embodiments of the presentdisclosure may include at least one of, for example, a smart phone, atablet personal computer (PC), a mobile phone, a video phone, anelectronic book reader (e-book reader), a desktop PC, a laptop PC, anetbook computer, a workstation, a server, a personal digital assistant(PDA), a portable multimedia player (PMP), a MPEG-1 audio layer-3 (MP3)player, a mobile medical device, a camera, and a wearable device. Thewearable device may include at least one of an accessory type (e.g., awatch, a ring, a bracelet, an anklet, a necklace, a glasses, a contactlens, or a head-mounted device (HMD)), a fabric or clothing integratedtype (e.g., an electronic clothing), a body-mounted type (e.g., a skinpad, or tattoo), and a bio-implantable type (e.g., an implantablecircuit).

The electronic device may be a home appliance. The home appliance mayinclude at least one of, for example, a television, a digital versatiledisc (DVD) player, an audio, a refrigerator, an air conditioner, avacuum cleaner, an oven, a microwave oven, a washing machine, an aircleaner, a set-top box, a home automation control panel, a securitycontrol panel, a television (TV) box (e.g., Samsung HomeSync™, AppleTV™, or Google TV™), a game console (e.g., Xbox™ and PlayStation™), anelectronic dictionary, an electronic key, a camcorder, and an electronicphoto frame.

The electronic device may include at least one of various medicaldevices (e.g., various portable medical measuring devices (a bloodglucose monitoring device, a heart rate monitoring device, a bloodpressure measuring device, a body temperature measuring device, etc.), amagnetic resonance angiography (MRA), a magnetic resonance imaging(MRI), a computed tomography (CT) machine, and an ultrasonic machine), anavigation device, a global positioning system (GPS) receiver, an eventdata recorder (EDR), a flight data recorder (FDR), a vehicleinfotainment devices, an electronic devices for a ship (e.g., anavigation device for a ship, and a gyro-compass), avionics, securitydevices, an automotive head unit, a robot for home or industry, anautomatic teller's machine (ATM) in banks, point of sales (POS) in ashop, or internet device of things (e.g., a light bulb, various sensors,electric or gas meter, a sprinkler device, a fire alarm, a thermostat, astreetlamp, a toaster, a sporting goods, a hot water tank, a heater, aboiler, etc.).

The electronic device may include at least one of a part of furniture ora building/structure, an electronic board, an electronic signaturereceiving device, a projector, and various kinds of measuringinstruments (e.g., a water meter, an electric meter, a gas meter, and aradio wave meter). In various embodiments, the electronic device may bea combination of one or more of the aforementioned various devices. Theelectronic device may also be a flexible device. The electronic deviceaccording to an embodiment of the present disclosure is not limited tothe aforementioned devices, and may include a new electronic deviceaccording to the development of technology.

Hereinafter, an electronic device according to various embodiments ofthe present disclosure will be described with reference to theaccompanying drawings. In the present disclosure, the term “user” mayindicate a person using an electronic device or a device (e.g., anartificial intelligence electronic device) using an electronic device.

FIG. 1 illustrates an electronic device within a network environment,according to an embodiment of the present disclosure.

Referring to FIG. 1, the electronic device 101 may operate within anetwork environment 100. The electronic device 101 may include a bus110, a processor 120, a memory 130, an input/output interface 150, adisplay 160, and a communication module 170. In some embodiments, theelectronic device 101 may omit at least one of the elements, or mayfurther include other elements.

The bus 110 may include, for example, a circuit that interconnects thecomponents 110 to 170 and delivers communications (for example, acontrol message and/or data) between the components 110 to 170.

The processor 120 may include one or more of a CPU, an AP, and acommunication processor (CP). The processor 120, for example, may carryout operations or data processing relating to the control and/orcommunication of at least one other element of the electronic device101.

The memory 130 may include a volatile and/or non-volatile memory. Thememory 130 may store, for example, instructions or data relevant to atleast one other element of the electronic device 101. According to anembodiment, the memory 130 may store software and/or a program 140. Theprogram 140 may include, for example, a kernel 141, middleware 143, anapplication programming interface (API) 145, and/or application programs(or “applications”) 147. At least some of the kernel 141, the middleware143, and the API 145 may be referred to as an operating system (OS).

The kernel 141 may control or manage the system resources (for example,the bus 110, the processor 120, the memory 130, etc.) that are used toexecute operations or functions implemented in the other programs (forexample, the middleware 143, the API 145, and the application programs147). Furthermore, the kernel 141 may provide an interface through whichthe middleware 143, the API 145, or the application programs 147 mayaccess the individual elements of the electronic device 101 to controlor manage the system resources.

The middleware 143 may function as an intermediary for allowing the API145 or the application programs 147 to communicate with the kernel 141to exchange data.

The middleware 143 may process one or more task requests, which arereceived from the application programs 147, according to prioritiesthereof. For example, the middleware 143 may assign priorities for usingthe system resources (for example, the bus 110, the processor 120, thememory 130, etc.) of the electronic device 101 to one or more of theapplication programs 147. The middleware 143 may perform scheduling orloading balancing on the one or more task requests by processing the oneor more task requests according to the priorities assigned to the one ormore application programs.

The API 145, which is an interface through which the applications 147control functions provided from the kernel 141 or the middleware 143,may include at least one interface or function (e.g., instruction) forfile control, window control, image processing, text control, etc.

The input/output interface 150 may function as an interface that canforward instructions or data, which are input from a user or an externaldevice, to the other element(s) of the electronic device 101.Furthermore, the input/output interface 150 may output the instructionsor data received from the other component element(s) of the electronicdevice 101 to the user or another external device.

Examples of the display 160 may include a liquid crystal display (LCD),a light-emitting diode (LED) display, an organic light-emitting diode(OLED) display, a micro-electro-mechanical systems (MEMS) display, andan electronic paper display. The display 160 may display various typesof content (e.g., text, images, videos, icons, symbols, etc.) for auser. The display 160 may include a touch screen and may receive atouch, gesture, proximity, or hovering input using an electronic pen orthe user's body part.

The communication module 170 may configure communication between theelectronic device 101 and an external device (e.g., a first externalelectronic device 102, a second external electronic device 104, or aserver 106). The communication module 170 may be connected to a network162 through wireless or wired communication to communicate with theexternal device (for example, the second external electronic device 104or the server 106).

The wireless communication may use at least one of long term evolution(LIE), LTE-Advance (LTE-A), code division multiple access (CDMA),Wideband CDMA (WCDMA), universal mobile telecommunications system(UNITS), WiBro (Wireless Broadband), Global System for MobileCommunications (GSM), and the like, as a cellular communicationprotocol. In addition, the wireless communication may include shortrange communication 164. The short range communication 164 may include,for example, at least one of Wi-Fi, Bluetooth (BT), near fieldcommunication (NFC), global navigation satellite system (GNSS), etc. TheGNSS may include at least one of a global positioning system (GPS), aglobal navigation satellite System (Glonass), a Beidou NavigationSatellite System (hereinafter referred to as “Beidou”), and a EuropeanGlobal Satellite-based Navigation System (Galileo), according to a usearea, a bandwidth, or the like. In the present disclosure, “GPS” may beinterchangeably used with “GNSS”. The wired communication may include atleast one of a universal serial bus (USB), a high definition multimediainterface (HDMI), Recommended Standard 232 (RS-232), a plain oldtelephone service (POTS), etc. The network 162 may include atelecommunication network, such as at least one of a computer network(e.g., a LAN or a WAN), the Internet, and a telephone network.

Each of the first and second external electronic devices 102 and 104 maybe of the same or a different type from the electronic device 101. Theserver 106 may include a group of one or more servers. All or some ofthe operations executed in the electronic device 101 may be executed inanother electronic device or a plurality of electronic devices (e.g.,the electronic devices 102 and 104 or the server 106). When theelectronic device 101 has to perform some functions or servicesautomatically or in response to a request, the electronic device 101 mayrequest another device (e.g., the electronic device 102 or 104 or theserver 106) to perform at least some functions relating thereto insteadof, or in addition to, performing the functions or services by itself.The other electronic device may perform the requested functions or theadditional functions and may transfer the execution result to theelectronic device 101. The electronic device 101 may provide thereceived result as it is, or may additionally process the receivedresult to provide the requested functions or services. To this end, forexample, cloud computing, distributed computing, or client-servercomputing technology may be used.

FIG. 2 is a block diagram of an electronic device 201 according to anembodiment of the present disclosure.

Referring to FIG. 2, the electronic device 201 may include all or partof the electronic device 101 illustrated in FIG. 1. The electronicdevice 201 may include at least one AP 210, a communication module 220,a subscriber identification module (SIM) card 224, a memory 230, asensor module 240, an input device 250, a display 260, an interface 270,an audio module 280, a camera module 291, a power management module 295,a battery 296, an indicator 297, and a motor 298.

The processor 210 may drive an OS or application programs to control aplurality of hardware or software elements connected thereto and mayperform various types of data processing and operations. The processor210 may be embodied as a system on chip (SoC). The processor 210 mayfurther include a graphic processing unit (GPU) and/or an image signalprocessor. The processor 210 may also include at least some of theelements illustrated in FIG. 2 (e.g., the cellular module 221). Theprocessor 210 may load, in a volatile memory, instructions or datareceived from at least one of the other elements (e.g., a non-volatilememory) to process the loaded instructions or data, and may storevarious types of data in the non-volatile memory.

The communication module 220 may have a configuration equal or similarto that of the communication module 170 of FIG. 1. The communicationmodule 220 may include a cellular module 221, a Wi-Fi module 223, a BTmodule 225, a GNSS module 227 (for example, a GPS module, a Glonassmodule, a Beidou module, or a Galileo module), an NFC module 228, and aradio frequency (RF) module 229.

The cellular module 221 may provide, for example, a voice call, a videocall, a text message service, an Internet service, etc. through acommunication network. According to an embodiment, the cellular module221 may identify and authenticate the electronic device 201 within acommunication network using the subscriber identification module 224(for example, a SIM card). The cellular module 221 may perform at leastsome of the functions that the processor 210 may provide. The cellularmodule 221 may include a CP.

The Wi-Fi module 223, the BT module 225, the GNSS module 227, or the NFCmodule 228 may each include a processor for processing data that istransmitted and received through the corresponding module. At least twoor more of the cellular module 221, the Wi-Fi module 223, the BT module225, the GNSS module 227, and the NFC module 228 may be included in oneintegrated chip (IC) or IC package.

The RF module 229 may transmit/receive a communication signal (forexample, an RF signal). The RF module 229 may include a transceiver, apower amplifier module (PAM), a frequency filter, a low noise amplifier(LNA), an antenna, etc. According to another embodiment of the presentdisclosure, at least one of the cellular module 221, the Wi-Fi module223, the BT module 225, the GNSS module 227, and the NFC module 228 maytransmit/receive an RF signal through a separate RF module.

The subscriber identification module 224 may include a card including asubscriber identity module and/or an embedded SIM, and may containunique identification information (e.g., an integrated circuit cardidentifier (ICCID)) or subscriber information (e.g., an internationalmobile subscriber identity (IMSI)).

The memory 230 (e.g., the memory 130) may include an internal memory 232and/or an external memory 234. The internal memory 232 may include atleast one of a volatile memory (e.g., a dynamic random access memory(DRAM), a static RAM (SRAM), a synchronous dynamic RAM (SDRAM), and thelike) and a non-volatile memory (e.g., a one time programmable read onlymemory (OTPROM), a programmable ROM (PROM), an erasable and programmableROM (EPROM), an electrically erasable and programmable ROM (EEPROM), amask ROM, a flash ROM, a flash memory (e.g., a NAND flash memory, a NORflash memory, and the like), a hard disc drive, a solid state drive(SSD), and the like).

The external memory 234 may further include a flash drive, such as acompact flash (CF), a secure digital (SD), a micro secure digital(Micro-SD), a mini secure digital (Mini-SD), an eXtreme digital (xD), amulti-media card (MMC), a memory stick, and the like. The externalmemory 234 may be functionally and/or physically connected to theelectronic device 201 through various interfaces.

The sensor module 240 may measure a physical quantity or detect theoperating state of the electronic device 201 and may convert themeasured or detected information into an electrical signal. The sensormodule 240 may include, for example, at least one of a gesture sensor240A, a gyro sensor 240B, an atmospheric pressure sensor 240C, amagnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, aproximity sensor 240G, a color sensor 240H (for example, a red, green,and blue (RGB) sensor), a biometric sensor 240I, a temperature/humiditysensor 240J, an illumination sensor 240K, and a ultraviolet (UV) sensor240M. Additionally or alternatively, the sensor module 240 may include,for example, an E-nose sensor, an electromyography (EMG) sensor, anelectroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, aninfrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. Thesensor module 240 may further include a control circuit for controllingone or more sensors included therein. In some embodiments, theelectronic device 201 may further include a processor configured tocontrol the sensor module 240 as a part of, or separately from, theprocessor 210, and may control the sensor module 240 while the processor210 is in a sleep state.

The input device 250 may include a touch panel 252, a (digital) pensensor 254, a key 256, or an ultrasonic input device 258. The touchpanel 252 may use at least one of a capacitive type, a resistive type,an infrared type, and an ultrasonic type. Furthermore, the touch panel252 may further include a control circuit. The touch panel 252 mayfurther include a tactile layer to provide a tactile reaction to a user.

The (digital) pen sensor 254 may include a recognition sheet that is apart of, or separate from, the touch panel. The key 256 may include aphysical button, an optical key, or a keypad. The ultrasonic inputdevice 258 may detect ultrasonic waves, which are generated by an inputtool, through a microphone (e.g., a microphone 288) to identify datacorresponding to the detected ultrasonic waves.

The display 260 (e.g., the display 160) may include a panel 262, ahologram device 264 or a projector 266. The panel 262 may have aconfiguration that is the same as, or similar to, that of the display160 illustrated in FIG. 1. The panel 262 may be implemented to beflexible, transparent, or wearable. The panel 262, together with thetouch panel 252, may be implemented as one module. The hologram device264 may show a three-dimensional image in the air by using aninterference of light. The projector 266 may display an image byprojecting light onto a screen. The screen may be located in theinterior of, or on the exterior of, the electronic device 201. Thedisplay 260 may further include a control circuit for controlling thepanel 262, the hologram device 264, or the projector 266.

The interface 270 may include a high-definition multimedia interface(HDMI) 272, a USB 274, an optical interface 276, or a D-subminiature(D-sub) 278. The interface 270 may be included in the communicationinterface 170 illustrated in FIG. 1. Additionally or alternatively, theinterface 270 may include a mobile high-definition link (MHL) interface,an SD card/MMC interface, or an infrared data association (IrDA)standard interface.

The audio module 280 may convert a sound into an electrical signal, andvice versa. At least some elements of the audio module 280 may beincluded in the input/output interface 150 illustrated in FIG. 1. Theaudio module 280 may process sound information that is input or outputthrough a speaker 282, a receiver 284, earphones 286, the microphone288, etc.

The camera module 291 is a device that can photograph a still image anda moving image. The camera module 291 may include one or more imagesensors (e.g., a front sensor or a rear sensor), a lens, an image signalprocessor (ISP), or a flash (e.g., an LED or xenon lamp).

The power management module 295 may manage the power of the electronicdevice 201. The electronic device 201 may be an electronic device thatreceives power through a battery, but the electronic device according tothe present disclosure is not limited thereto. The power managementmodule 295 may include a power management integrated circuit (PMIC), acharger integrated circuit (IC), or a battery or fuel gauge. The PMICmay have a wired and/or wireless charging method. Examples of thewireless charging method may include a magnetic resonance method, amagnetic induction method, an electromagnetic wave method, etc.Additional circuits (e.g., a coil loop, a resonance circuit, arectifier, etc.) for wireless charging may be further included. Thebattery gauge may measure the residual amount of the battery 296 as wellas a voltage, current, or temperature while charging. The battery 296may include a rechargeable battery and/or a solar battery.

The indicator 297 may indicate a particular state (e.g., a bootingstate, a message state, a charging state, and the like) of theelectronic device 201 or a part (for example, the processor 210)thereof. The motor 298 may convert an electrical signal into amechanical vibration and may generate a vibration, a haptic effect, andthe like. Although not illustrated, the electronic device 201 mayinclude a processing unit (e.g., a GPU) for supporting mobile TV. Theprocessing unit for supporting mobile TV may process media dataaccording to a standard, such as digital multimedia broadcasting (DMB),digital video broadcasting (DVB), mediaFlo™, and the like.

Each of the above-described component elements of hardware according toembodiments of the present disclosure may be configured with one or morecomponents, and the names of the corresponding component elements mayvary based on the type of electronic device. The electronic deviceaccording to various embodiments of the present disclosure may includeat least one of the aforementioned elements. Some elements may beomitted or other additional elements may be further included in theelectronic device. Also, some of the hardware components according tovarious embodiments may be combined into one entity, which may performfunctions identical to those of the relevant components before thecombination.

FIG. 3 is a block diagram of a program module according to an embodimentof the present disclosure.

Referring to FIG. 3, the program module 310 (e.g., the program 140) mayinclude an OS that controls resources relating to an electronic device(e.g., the electronic device 101) and/or various applications (e.g., theapplication programs 147) that are executed in the OS. The OS may be,for example, Android™, iOS™, Windows™ Symbian™, Tizen™, Bada™, and thelike.

The program module 310 may include a kernel 320, middleware 330, an API360, and/or applications 370. At least a part of the program module 310may be preloaded on the electronic device, or may be downloaded from anexternal electronic device (e.g., the electronic device 102 or 104 orthe server 106).

The kernel 320 (e.g., the kernel 141) may include a system resourcemanager 321 and/or a device driver 323. The system resource manager 321may control, allocate, or retrieve system resources. The system resourcemanager 321 may include a process manager, a memory manager, or a filesystem manager. The device driver 323 may include a display driver, acamera driver, a BT driver, a shared memory driver, a USB driver, akeypad driver, a Wi-Fi driver, an audio driver, or an inter-processcommunication (IPC) driver.

The middleware 330 may provide a function required by the applications370 in common, or may provide various functions to the applications 370through the API 360 such that the applications 370 can efficiently uselimited system resources within the electronic device. The middleware330 (e.g., the middleware 143) may include, for example, at least one ofa runtime library 335, an application manager 341, a window manager 342,a multimedia manager 343, a resource manager 344, a power manager 345, adatabase manager 346, a package manager 347, a connectivity manager 348,a notification manager 349, a location manager 350, a graphic manager351, and a security manager 352.

The runtime library 335 may include a library module that a compileruses in order to add a new function through a programming language whilethe applications 370 are being executed. The runtime library 335 mayperform input/output management, memory management, the functionalityfor an arithmetic function, and the like.

The application manager 341 may manage the life cycle of at least one ofthe applications 370. The window manager 342 may manage graphical userinterface (GUI) resources used on a screen. The multimedia manager 343may determine formats required to reproduce various media files and mayencode or decode a media file using a coder/decoder (codec) appropriatefor the corresponding format. The resource manager 344 may manageresources, such as the source code, the memory, the storage space, andthe like of at least one of the applications 370.

The power manager 345 may operate together with, for example, a basicinput/output system (BIOS) to manage a battery or power and providepower information required for the operation of the electronic device.The database manager 346 may generate, search, and/or change a databaseto be used by at least one of the applications 370. The package manager347 may manage the installation or update of an application that isdistributed in the form of a package file.

The connectivity manager 348 may manage a wireless connection, such asWi-Fi, BT, and the like. The notification manager 349 may display ornotify of an event, such as an arrival message, an appointment, aproximity notification, etc., in such a manner as not to disturb a user.The location manager 350 may manage the location information of theelectronic device. The graphic manager 351 may manage a graphic effectto be provided to a user and a user interface relating to the graphiceffect. The security manager 352 may provide various security functionsrequired for system security, user authentication, etc. In a case wherethe electronic device (e.g., the electronic device 101) has a telephonecall function, the middleware 330 may further include a telephonymanager for managing a voice or video call function of the electronicdevice.

The middleware 330 may include a middleware module that forms acombination of various functions of the above-described elements. Themiddleware 330 may provide specialized modules according to the types ofOS in order to provide differentiated functions. Furthermore, themiddleware 330 may dynamically remove some of the existing elements, ormay add new elements.

The API 360 (e.g., the API 145) is a set of API programming functions,and may be provided with different configurations according to OS. Forexample, in the case of Android or iOS, one API set may be provided foreach platform, and in the case of Tizen, two or more API sets may beprovided for each platform.

The applications 370 (e.g., the application programs 147) may includeone or more applications that can perform functions, for example, home371, dialer 372, short message service (SMS)/multi-media message service(MMS) 373, instant message (IM) 374, browser 375, camera 376, alarm 377,contacts 378, voice dial 379, e-mail 380, calendar 381, media player382, album 383, clock 384, health care (e.g., measuring exercisequantity or blood sugar), and environment information (e.g., atmosphericpressure, humidity, temperature information, and the like).

The applications 370 may include an application (“information exchangeapplication”) that supports information exchange between the electronicdevice (e.g., the electronic device 101) and an external electronicdevice (e.g., the electronic device 102 or 104). The informationexchange application may include a notification relay application forforwarding specific information to the external electronic device or adevice management application for managing the external electronicdevice.

The notification relay application may include a function oftransferring, to the external electronic device (e.g., the electronicdevice 102 or 104), notification information that is generated from theother applications (e.g., the SMS/MMS application, the e-mailapplication, the health care application, the environmental informationapplication, and the like) of the electronic device. Furthermore, thenotification relay application may receive notification information fromthe external electronic device and may provide the received notificationinformation to a user.

The device management application may manage (e.g., install, delete, orupdate), at least one function of an external electronic device (e.g.,the electronic device 102 or 104) that communicates with the electronicdevice (e.g., a function of turning on/off the external electronicdevice itself (or some components thereof) or a function of adjustingthe brightness (or resolution) of a display), applications that operatein the external electronic device, or services (e.g., a call service, amessage service, and the like) that are provided by the externalelectronic device.

The applications 370 may include applications (e.g., a health careapplication of a mobile medical appliance, and the like) that arespecified according to attributes of an external electronic device(e.g., the electronic device 102 or 104). The applications 370 mayinclude applications that are received from an external electronicdevice (e.g., the server 106, or the electronic device 102 or 104). Theapplications 370 may include preloaded applications or third-partyapplications that can be downloaded from a server. Names of the elementsof the program module 310, according to the above-described embodimentsof the present disclosure, may change depending on the type of OS.

According to various embodiments of the present disclosure, at leastsome of the program module 310 may be implemented in software, firmware,hardware, or a combination of two or more thereof. At least a part ofthe program module 310 may be implemented (e.g., executed) by, forexample, a processor (e.g., the processor 210). At least some of theprogram module 310 may include, for example, a module, a program, aroutine, a set of instructions, and/or a process for performing one ormore functions.

FIG. 4 illustrates an example of an electronic device that outputs imagedata according to an embodiment of the present disclosure.

Referring to FIG. 4, the electronic device 101 may include, for example,a lens 420, an image sensor 430 that detects an object 410 through thelens 420, a controller 440, and the display 160 that outputs an image.

The image sensor 430 may generate image data in accordance with anoptical signal. The controller 440 may process the generated image datato display the processed image data on the display 160 that isfunctionally connected to the electronic device. The controller 440 mayidentify a mode for generating image data corresponding to the imagedetected by the image sensor 430 and determine a setting for at leastone image attribute (e.g., frame rate and/or resolution) to be used forgenerating the image at least based on the mode.

The image sensor 430 may be configured in the unit of a plurality ofpixels, and each unit pixel may include a plurality of subpixels. Thesubpixel may include a photodiode. The image sensor 430 may include astructure having two or more subpixels (e.g., photodiodes) in one unitpixel. The image sensor 430 may output color information including atleast one of color information among R (red), G (green), and B (blue).

The image sensor 430 may include a pixel array 431 including a pluralityof subpixels, a row driver 432 that controls an array of the subpixelsin the unit or rows, a lead-out 433 that stores, senses, amplifies,and/or outputs a signal output from the array of the subpixels, a timinggenerator 434 that provides a clock signal to the row driver 432 and thelead-out 433, and/or a control register 435 that stores various commandsrequired for the operation of the image sensor 430. The controller 440may include a camera controller 441, an image signal processor 442,and/or an I/F (interface) 443.

The image sensor 430 may detect the object 410 imaged through the lens420 by a control of the controller 440. The controller 440 may outputimage data detected and generated by the image senor 430 on the display160. The display 160 may be any device that may output image data. Forexample, the display 160 may be implemented as a dedicated display (suchas a monitor or a display formed on the electronic device, such as acomputer, a mobile phone, a TV, or a camera). The image sensor 430 mayacquire an optical signal corresponding to the object 410. The imagesensor 430 may be functionally connected to the controller 440. Althoughthe image sensor 430 is described as an element of the image sensor orthe camera, the image sensor 430 is not limited thereto and can bevariously modified.

The controller 440 may output information for calculating a phasedifference between two or more subpixels. Further, the controller 440may obtain color information by using information read from the imagesensor 430. The controller 440 may identify a setting of imageattributes corresponding to the object 410. For example, when thesetting of the image attributes meets a predetermined condition (e.g.,the setting is within a first predetermined range), the image sensor 430may output pixel data of an image corresponding to the object 410 byusing a first signal corresponding to a first subpixel and a secondsignal corresponding to a second subpixel. Further, the image sensor 430may generate image data by using the pixel data. In addition, when thesetting of the image attributes does not meet the predeterminedcondition (e.g., the setting is within another predetermined range(e.g., a second predetermined range) that is different from the firstpredetermined range), the controller 440 may control (or set) thecorresponding pixel to be not used for determining the phase differencebased on the first signal corresponding to the first subpixel and thesecond signal corresponding to the second subpixel.

The controller 440 may adjust information (e.g., information forcalculating the phase difference or image data) output through the imagesensor 430. For example, the controller 440 may differently adjust theinformation output through the image sensor 430 based on the setting ofthe image attributes or the mode (e.g., first mode to fourth mode). Thecontroller 440 may differently adjust the information (e.g., informationfor calculating the phase difference or image data) output through theimage sensor 430 according to each mode. For example, in the first mode,the controller 440 may control the image sensor 430 to output only theinformation for calculating the phase difference by using a signal ofeach subpixel level. Further, the controller 440 may identify phasedifference information by post-processing the information forcalculating the phase difference.

In the second mode, the controller 440 may control the image sensor 430to output image data generated by combining data of respective subpixellevels (e.g., the first signal and the second signal). For example, thesignal acquired through the image sensor 430 may include one unit pixellevel signal that is not identified as the first signal or the secondsignal.

In the third mode, the controller 440 may control the image sensor 430to output information for calculating a phase difference by using thesignal of each subpixel level. Further, the controller 440 may controlthe image sensor 430 to process and output data of each subpixel level(e.g., the first signal and the second signal) as image data of the unitpixel level (e.g., combine or average the first signal and the secondsignal).

In the fourth mode, the controller 440 may control the image sensor 430to output information for calculating the phase difference by using thesignal of each subpixel level. Further, the controller 440 may controlthe image sensor 430 to process and output data of each subpixel level(e.g., the first signal and the second signal) as image data of the unitpixel level (e.g., combine or average the first signal and the secondsignal). For example, the controller 440 may differently adjust anoperation for processing the image data based on attributes (e.g.,color) of the unit pixel. When the attributes of the unit pixelcorrespond to red or blue, the controller 440 may control the imagesensor 430 to output image data generated by combining the data ofrespective subpixel levels (e.g., first signal and second signal).Further, when the attributes of the unit pixel correspond to green, thecontroller 440 may control the image sensor 430 to process and outputthe data of the respective subpixel levels (e.g., first signal andsecond signal) as the image data of the unit pixel level

The controller 440 may determine a setting of the image attributes basedon a user input as well the setting of the image attributes. Forexample, when the user input is generated, the controller 440 mayidentify a mode corresponding to the user input and determine a settingof image attributes related to the mode. When there is a user inputreceived through a camera (e.g., the camera 376) in applicationsincluded in the electronic device 101, the controller 440 may identify adetermined mode through the user input. Further, the controller 440 maydetermine a setting of the image attributes based on the mode.

When the setting of the image information meets a predeterminedcondition (e.g., the setting is within a first predetermined range), thecontroller 440 may control (or set) the corresponding pixel to generateimage data by using the first signal and the second signal.

When the setting of the image information does not meet thepredetermined condition (e.g., the setting is within a predeterminedrange (e.g., second predetermined range) different from the firstpredetermined range), the controller 440 may control the image sensor430 to generate the image data by reading a combination of the firstsignal and the second signal. The image information may include, forexample, a size of the object 410, a distance between the object 410 andthe image sensor, a movement speed of the object 410, a frame rate ofthe image to be output through the display 160, or a combinationthereof.

When the image information includes the frame rate of the image to beoutput, the predetermined condition (e.g., first predetermined range) ofthe setting of the image information may include a low speed frame rate,and the other predetermined range (e.g., second predetermined range)that does not meet the predetermined condition may include a high speedframe rate. The low speed frame rate may include 30 frames per second,and the high speed frame rate may include 60 frames per second. The highspeed frame rate and the low speed frame rate, according to variousembodiments, may vary depending on the specification or technologydevelopment of the electronic device 101.

The camera controller 441 may control the control register 435. Thecamera controller 441 may control the image sensor 430 or the controlregister 435 by using, for example, an Inter-Integrated Circuit (I2C).In some embodiments, the controller 440 may omit at least one of theelements or may additionally include another element.

The image signal processor 442 may receive a sub-pixel signal (SPS)corresponding to an output signal of the lead-out 433 and process and/orhandle the subpixel signal in the unit of subpixels or pixels, so as togenerate image data. The image signal processor 442 may display, forexample, the image data on the display 160 through the I/F 443.

The pixel array 431 may include a plurality of unit pixels. Each unitpixel may include a plurality of subpixels. Each unit pixel may includetwo photodiodes or four photodiodes. Each subpixel may sense lightincident through the lens 420 and output at least one subpixel signalaccording to a control of the row driver 432. The subpixels signal maybe a digital signal having a value of “0” or “1”. An example of thesubpixel will be described in detail with reference to FIGS. 5, and 6Ato 6E.

The timing generator 434 may control the operation or timing of the rowdriver 432 and/or the lead-out 433 by outputting a control signal or aclock signal to each of the row driver 432 and/or the lead-out 433. Thecontrol register 435 operates according to a control of the cameracontroller 441, stores various commands required for the operation ofthe image sensor 430, and transmits the various commands to the timinggenerator 434.

The pixel array 431 may output the subpixel signal from a row selectedby each control signal provided by the row driver 432 to the lead-out433.

The lead-out 433 may temporarily store the subpixel signal output fromthe pixel array 431, and then sense, amplify, and output the subpixelsignal. The lead-out 433 may include a plurality of column memories(e.g., SRAM) included in each column to temporarily store the pixelsignal, a sense amplifier (SA) (not shown) for sensing and amplifyingthe pixel signal, or a counter (not shown) for counting the temporarilystored pixel signal.

The lens 420 may include a main lens. For example, the main lens may beimplemented in the size corresponding to the whole pixel array 431 andmake an image of the object 410 focused.

The controller 440 may identify a mode for outputting image datacorresponding to the object detected through the image sensor 430, anddetermine a setting for at least one image attribute (e.g., frame rateand/or resolution) to be used for generating the image at least based onthe identified mode. Further, the image sensor 430 may generate theimage data by using pixel data corresponding to an optical signal atleast based on the setting. The mode may include a first mode in whichsignals (or pixel data corresponding to optical signals) of a subpixelslevel are output from all subpixels included in the image sensor 430, asecond mode in which image data of a unit pixel level is output bycombining and reading signals of each subpixel level within the unitpixel, a third mode in which signals are acquired from respectivesubpixels within the unit pixel, information for calculating a phasedifference between subpixels is generated through the acquired signals,and image data of a unit pixel level generated by combining or averagingsignals acquired from respective subpixels within the unit pixel and theinformation for calculating the phase different are output, and a fourthmode in which image data of the unit pixel level is output by combiningand reading signals of respective subpixels within a first unit pixel(e.g., R pixel or B pixel) at the subpixel level, information forcalculating a phase different is output using signals acquired fromrespective subpixels within a second unit pixel (e.g., G pixel), andimage data of the unit pixel level is output by combining or averagingthe acquired signals at the subpixel level. In the fourth mode, theimage data output by the first unit pixel and the image data and theinformation output by the second unit pixel may be output together.Further, the controller 440 may control the electronic device 101 todisplay the image data output by the image sensor 430 on the display 160through a post-processing process.

The controller 440 may identify a phase difference by using informationfor calculating the phase difference. For example, based on a signal(e.g., information for calculating the phase difference) acquired ineach subpixel, the controller 440 may identify attributes (e.g.,brightness information or color information) included in the signal andidentify the phase difference based on a difference between theidentified attributes (e.g., a difference of the attributes of thesubpixels). The controller 440 may identify the phase difference basedon a difference between brightness information acquired in thesubpixels.

The electronic device 101 (controller 440) may output signals ofsubpixel levels in all subpixels of the image sensor 430, for example,in the first mode. The image sensor 430 may output the signal of thesubpixel level in each of the plurality of subpixels included in eachunit pixel.

The electronic device 101 (controller 440) may output image data of aunit pixel level corresponding to a sum of the subpixels according toeach unit pixel of the image sensor 430 in order to generate the imagein the second mode at a higher speed than that in the first mode. Forexample, since an amount of data becomes smaller in the second modecompared to the first mode, the electronic device 101 (controller 440)may output the image data at a high speed and may not output theinformation for calculating the phase difference between subpixels.

The electronic device 101 (controller 440) may acquire a signal in eachsubpixel within the unit pixel of the image sensor 430, outputinformation for calculating a phase difference between subpixels byusing the acquired signal, combine or average the signals of thesubpixels according to each unit pixel, read the added up or averagedsignals, and output image data of the unit pixel level in the thirdmode. In the third mode, the information and the image data may besimultaneously output. The information may include filtered data forcalculating the phase difference.

The electronic device 101 (controller 440) may combine and read signalsacquired from respective subpixels within the first unit pixel at thesubpixel level and output image data of the unit pixel level. Further,signals may be acquired from respective subpixels within the second unitpixel, information for calculating a phase difference between subpixelswithin the second unit pixel may be generated using the acquiredsignals, and image data of the unit pixel level may be output bycombining or averaging and reading the acquired signals. Further, in thefourth mode, the image data output by the first unit pixel, the imagedata output by the second unit pixel, and the generated information maybe combined and output. The electronic device 101 (controller 440) maycombine and output signals of respective subpixels in the correspondingpixel between the R pixel and the B pixel and output each signal of thesubpixel level from each subpixel of the G pixel in the fourth mode. Inthis case, in the fourth mode, information for calculating a phasedifference between subpixels and the image data may be simultaneouslyoutput. In the fourth mode, for example, information for calculating aphase difference between subpixels of the G pixel may be output.

In the first mode, the electronic device 101 (controller 440) may setimage information as AF and set the setting of the image information ashigh speed AF (for example, AF using phase difference information). Inthe first mode, the controller 440 may output image data, for example,at a low frame rate (for example, 30 frames per second). In the firstmode, the controller 440 may output a signal of a subpixel level in eachunit pixel.

In the second mode, the electronic device 101 (controller 440) may setimage information as a frame rate and set the setting of the imageinformation as a high frame rate. In the second mode, the controller 440may output the image data, for example, a high frame rate (for example,60 frames per second) and may not generate information for calculating aphase difference between subpixels. The controller 440 may output onlyimage data of the unit pixel level by combining signals of respectivesubpixels within each unit pixel for a high speed operation.

In the third mode, the electronic device 101 (controller 440) may setimage information as AF and frame rate, and may set the setting of AF ashigh speed AF (for example, AF using phase difference information) andthe setting of frame rate as high frame rate. In the third mode, thecontroller 440 may acquire signals in respective subpixels within theunit pixel, generate information for calculating a phase difference byusing the acquired signals, and combine or average the signals acquiredin the respective subpixels within the unit pixel at the subpixellevels, so as to output image data of the unit pixel level. In the thirdmode, the information and the data, which has been added up or averaged,may be output at the same time.

In the fourth mode, the electronic device 101 (controller 440) may setimage information as AF and frame rate, and set the setting of AF ashigh speed AF (e.g., AF using phase difference information) and thesetting of frame rate as high frame rate. In the fourth mode, thecontroller 440 may output image data at a higher frame rate (forexample, 60 frames per second) compared to the first mode. Further, thecontroller 440 may output image data by using power that is lower thanpower used to output the image data in the third mode. The controller440 may output image data of the unit pixel level by combining signalsacquired in respective subpixels within a first unit pixel (e.g., Rpixel and B pixel) at the subpixel level. The controller 440 may outputinformation for calculating a phase difference by using signals acquiredin respective subpixels within a second unit pixel (e.g., G pixel) andoutput image data of the unit pixel level by combining or averaging theacquired signals at the subpixel level. The controller 440 may outputthe image data output by the first unit pixel and the image data outputby the second unit pixel and the information for calculating the phasedifference in the fourth mode. The information for calculating the phasedifference between subpixels may be added to the generated image data.Further, the generated image data may be output along with theinformation. The first unit pixel may include the R pixel and the Bpixel, and the second unit pixel may include the G pixel. The controller440 may combine the signals acquired in subpixels included in the pixelcorresponding to the R pixel or the B pixel and combine or average thesignals acquired in subpixels included in the G pixel, so as to outputimage data of the unit pixel level. Further, the controller 440 maygenerate the information for calculating the phase difference by usingthe signals of the subpixels included in the G pixel.

The controller 440 may process the image data output from the R pixel,the image data output from the B pixel, and the image data output fromthe G pixel according to each mode and display the image on the display160. The controller 40 may process, in the unit of subpixels or unitpixels, the signals of the subpixel level output from the lead-out 433or the image data in the unit of unit pixels to generate the image anddisplay the generated image on the display 160.

Although it is illustrated that all the elements are included in theelectronic device 101, various embodiments are not limited thereto. Forexample, according to the role, function, or capability of theelectronic device 101, at least some of the elements of the electronicdevice 101 may be distributed to the electronic device 101 and anexternal electronic device (for example, the first external electronicdevice 102, the second external electronic device 104, or the server106).

According to various embodiments of the present disclosure, although thecontroller 440 is illustrated as an element separated from the imagesensor 430, various embodiments are not limited thereto. For example,the controller 440 may be included in at least a part of the imagesensor 430 according to a role, function, or capability of theelectronic device 101. For example, the camera controller 441 or theinterface unit (I/F) 443 may be included in the image sensor 430, andthe image signal processor 442 may be implemented to be separated fromthe image sensor 430.

According to various embodiments of the present disclosure, at leastsome of the camera controller 441, the image signal processor 442, orthe interface unit (I/F) 443 may be implemented in software, firmware,hardware, or a combination of two or more thereof. At least some of thecamera controller 441, the signal processor 442, or the interface unit(I/F) 443 may be implemented (for example, executed) by, for example, aprocessor (for example, the processor 120). At least some of the cameracontroller 441, the image signal processor 442, and the interface unit(I/F) 443 may include, for example, a module, a program, a routine, setsof instructions, or a process for performing one or more functions.

According to various embodiments of the present disclosure, theelectronic device 101 may implement, for example, the camera controller441, the image signal processor 442, or the I/F 443 to be integratedwith the processor (for example, the processor 120 of FIG. 1), and thecamera controller 441, the image signal processor 442, or the I/F 443may be stored, in a software form, in a (dedicated) memory areaaccessible by the processor and implemented to be executable by theprocessor.

FIG. 5 illustrates an example illustrating a configuration of a pixelarray of the image sensor according to an embodiment of the presentdisclosure.

Referring to FIG. 5, the pixel array 431 of the image sensor 430 mayinclude, for example, a plurality of unit pixels 510-a to 510-d arrangedin a matrix type, and each unit pixel may include a plurality ofsubpixels. The image sensor 430 may output signals corresponding to theproduct of all unit pixels and subpixels of each unit pixel at once oroutput signals corresponding to the number of all unit pixels bycombining signals of the subpixel level of one unit pixel. The firstunit pixel 510-a may include a plurality of subpixels 511, 512, 513, and514. The second unit pixel 510-b may include a plurality of subpixels515, 516, 517, and 518. Alternatively, the unit pixel may include twosubpixels (or photodiodes) or four subpixels (or photodiodes). Asubpixel pitch may be smaller than a pixel pitch in the general imagesensor 430.

Although FIG. 5 illustrates that each of the first unit pixel 510-a tothe fourth unit pixel 510-d includes four subpixels, the presentdisclosure is not limited thereto.

The pixel array 431 may output signals of respective subpixel levelsaccording to a column line in the unit of rows based on a control of thetiming generator 434.

A filter array (not shown) including each color filter for penetratingor blocking light of a particular spectrum area may be arranged on anupper part of the unit pixels included in the pixel array 431. Further,a micro lens for increasing, for example, light gathering power may bearranged on an upper part of a plurality of unit pixels included in thepixel array 431.

The row driver 432 may drive a plurality of control signals forcontrolling the operation of each of a plurality of subpixels 420 intothe pixel array 431 according to a control of the timing generator 434.For example, the plurality of control signals may include a signal forcontrolling transmission of optical charge generated by each of theplurality of subpixels, a signal for selecting each of the plurality ofsubpixels, or a signal for resetting each of the plurality of subpixels.The lead-out block 433 may include various elements (for example, acounter, a memory, a lead-out circuit, or a sense amp circuit) forprocessing signals of the subpixel level output from the pixel array431.

The image signal processor 442 may process and/or handle the signals ofthe subpixel level output from the first unit pixel 510-a to the fourthunit pixel 510-d. For example, the image signal processor 442 maycombine, for example, the signals of the subpixel level and addedsignals as one unit pixel level image data.

The lead-out 433 may output the signal of the subpixel levelcorresponding to the subpixel of each of the first unit pixel 510-a tothe fourth unit pixel 510-d. The first unit pixel 510-a to the fourthunit pixel 510-d may output, for example, signals of the subpixel levelcorresponding to a plurality of subpixels having passed through thecorresponding micro lens. The image signal processor 442 may processand/or handle, for example, the signals of the subpixel level andgenerate angular information and depth data.

FIG. 6A schematically illustrates a unit pixel having two photodiodes inthe image sensor according to an embodiment of the present disclosure.

FIG. 6B schematically illustrates a unit pixel having four photodiodesin the image sensor according to an embodiment of the presentdisclosure.

FIG. 6C illustrates an example of a structure of the unit pixel havingtwo photodiodes according to an embodiment of the present disclosure.

FIG. 6D is a circuit diagram illustrating the unit pixel having twophotodiodes in the image sensor according to an embodiment of thepresent disclosure.

FIG. 6E illustrates an example of a time chart of each of the switchesin the unit pixel according to an embodiment of the present disclosure.

Referring to FIGS. 6A to 6E, when light enters an optical conductorthrough a color filter, electron-positive hole generated in the opticalconductor is changed according to a wavelength and strength of thelight, and the image sensor 430 according to an embodiment of thepresent disclosure may output a voltage signal of a level at which thesignal can be processed. The image sensor 430 may be divided into, forexample, a charge coupled device (CCD) type image sensor and acomplementary metal oxide semiconductor (CMOS) type image sensoraccording to a type thereof. The image sensor 430 may configure aplurality of unit pixels, and may use an image sensor array, in which aplurality of unit pixels is arranged in predetermined columns and rows,in order to acquire a predetermined size of image data.

The unit pixel of the image sensor 430 may include two photodiodes 640,a color filter 630, and/or a micro lens 620. Further, the unit pixel ofthe image sensor 430 may include four photodiodes 650, the color filter630, and/or the micro lens 620. The image sensor 430, according to anembodiment of the present disclosure, may include an array of unitpixels consisting of a plurality of subpixels as illustrated in FIGS. 6Aand 6B. The number of subpixels per unit pixel may be predetermined, andFIG. 6A illustrates two subpixels and FIG. 6B illustrates foursubpixels.

The controller 440 may combine signals of respective subpixel levels ofthe unit pixel and output one image data. Further, the controller 440may output information for calculating a phase difference betweensubpixels (e.g., photodiodes) include in each unit pixel. For example,as illustrated in FIG. 6A, the unit pixel according to an embodiment ofthe present disclosure may also output information for calculating aphase difference of light incident to each photodiode and output colorinformation corresponding to a combination of signals of two subpixellevels (e.g., photodiodes).

The image sensor may output a value of each photodiode and, accordingly,one unit pixel may output each piece of color information of a pluralityof photodiodes. For example, when the image sensor having pixels of HD(1280×720) includes a 4PD structure having four photodiodes in one unitpixel, the image sensor may read a color value of each photodiode andoutput image data having resolution of quad high definition (QHD)(2560×1440).

The photodiodes 640 and 650 may convert incident light into an electricsignal, generate the converted electric signal, and include accumulateddoping areas. Further, the doping areas may be accumulated based on anincidence angle of the incident light.

The lens 610 may be functionally connected to an actuator for opticalimage stabilization (OIS) or auto focus (AF).

The color filter 630 may be an R (red) filter, a G (green) filter, a B(blue) filter, a yellow filter, a magenta filter, or a cyan filter.

The color filter 630 may be formed on the photodiodes 640 and 650 basedon the incidence angle of the incident light and have a Bayer pattern.In the Bayer pattern, filters that receive brightness of each of red,green, and blue on a two-dimensional plane in order to make image dataincluding dots by gathering brightness and colors of a target. Each unitpixel forming a grid network below the Bayer pattern color filter mayrecognize only a color allocated among red, green, and/or blue insteadof recognizing full color, and may interpolate the color so as to inferthe full color.

The micro lens 620 may be formed to correspond to the photodiodes 640and 650 on the color filter 630 while maintaining an angle ofinclination at which the color filter 630 is accumulated. Further, theOIS lens 610 may be located inside, a lens mount (not shown) and maygather light.

Referring to FIG. 6C, according to an embodiment of the presentdisclosure, the image sensor 430 may include a plurality of unit pixels,each unit pixel 660 may include two or more photodiodes, and a barrier670 may be located between the photodiodes. Further, at least one colorfilter 630 may be located on the plurality of photodiodes. In addition,at least one micro lens 620 may be located on the plurality ofphotodiodes. The micro lens 620 may be located on the color filter 630.Light incident to each unit pixel may be incident to each of differentphotodiodes via at least one micro lens and at least one color filter,and may output information for focus detection according to a phasedifference of the light incident to each photodiode.

Each unit pixel of the image sensor may include a photodiode (PD) fordetecting an amount of light, Transmission Switches TX_T1 and TX_T2 fortransmitting a charge generated by photodiodes D1 and D2 throughfloating diffusion, a selection switch RS for selecting a correspondingpixel among at least two pixels included in the image sensor, a resetswitch RST for resetting a floating diffusion area, and an amplificationswitch SF for outputting a potential of the floating diffusion area withan output voltage of the corresponding transmission switch.

Referring to FIGS. 6D and 6E, the two photodiodes D1 and D2 may beelectrically connected to, for example, the transmission switches TX_T1and TX_T2 665 and 667 for transmitting their own signals, respectively.

The photodiodes D1 and D2 may receive light having passed through thesame micro lens. For example, each of the two photodiodes may receivelight having passed through a color filter area and may generate acharge corresponding to received light energy.

The charge generated in the photodiodes D1 and D2 through thetransmission switches 665 and 667 may be transmitted to a floatingdiffusion area 669. The selection switch RS 661 may select acorresponding pixel from at least two pixels included in the imagesensor. The reset switch RST 663 may reset the floating diffusion area669. When one of the transmission switches 665 and 667 transmits thecharge to the floating diffusion area 669 after the reset switch 663 isreset, the amplification switch SF 671 may output the potential of thefloating diffusion area 669 with the output voltage of the correspondingtransmission switches 665 and 667.

When a color filter area of an optical filter is accumulated in thephotodiodes D1 and D2, the output voltage of the photodiodes D1 and D2may be components of color information output from the image sensor. Forexample, the output voltage of the photodiodes D1 may be a T1 signal ofFIG. 6E, and the output voltage of the photodiodes D2 may be a T2 signalof FIG. 6E.

A time chart of each of the switches 663, 665, and 671 will bedescribed. The selection switch 661 may be at a high level (hereinafterreferred to as an “active state”) for a predetermined time according tothe selection of the corresponding pixel. The reset switch 663 may resetthe floating diffusion area 669 by periodically generating a pulsehaving a high level for a predetermined time according to the activestate of the selection switch 661.

As the transmission switches 665 and 667 sequentially generate pulses(hereinafter referred to as a “transmission pulses”) having a high levelfor a predetermined time in a predetermined order according to thegeneration of the reset pulse by the selection switch 661, the chargesgenerated in the photodiodes D1 and D2 may be sequentially transmittedto the floating diffusion area 669. Further, one of the transmissionswitches 665 and 667 may transmit the charge to the floating diffusionarea 669 and the amplification switch 671 may output the potential ofthe floating diffusion area 669 with the output voltage of thecorresponding transmission switch.

The electronic device 101 according to an embodiment of the presentdisclosure may include an image sensor for acquiring an optical signalcorresponding to an object 410, a controller for identifying a mode forgenerating an image corresponding to the object 410 by using the opticalsignal, determining a setting of at least one image attribute to be usedfor generating the image at least based on the mode, generating imagedata by using pixel data corresponding to the optical signal at leastbased on the setting, and displaying the image corresponding to theobject 410 through a display functionally connected to the electronicdevice at least based on the image data, and a display for displayingthe image data.

According to various embodiments of the present disclosure, thecontroller 440 determines a first setting of the at least one imageattribute as the setting when the mode corresponds to a first mode,determines a second setting of the at least one image attribute as thesetting when the mode corresponds to a second mode, determines a thirdsetting of the at least one image attribute as the setting when the modecorresponds to a third mode, and determines a fourth setting of the atleast one image attribute as the setting when the mode corresponds to afourth mode.

According to various embodiments of the present disclosure, thecontroller may identify a first mode in which a plurality of subpixelsincluded in at least one of a plurality of unit pixels of the imagesenor functionally connected to the electronic device output pixel dataof a subpixel level, respectively, a second mode in which image data ofa unit pixel level is output through a combination of the pixel data ofthe plurality of subpixels within the unit pixel, a third mode in whichsignals are acquired from respective subpixels within the unit pixel,information for calculating a phase difference between the subpixelswithin the unit pixel is output, and image data of the unit pixel levelis output by combining or averaging the acquired signals at the subpixellevel, and a fourth mode in which image data of the unit pixel level isoutput by combining signals of respective subpixels within a first unitpixel (e.g., R pixel or B pixel), information for calculating a phasedifference using signals acquired from respective subpixels within asecond unit pixel (e.g., G pixel) is generated, and image data of theunit pixel level is output by combining or averaging the acquiredsignals at the subpixel level. Further, in the fourth mode, image dataoutput by a first unit pixel, image data output by a second unit pixel,and information for calculating the phase difference may be outputtogether.

According to various embodiments of the present disclosure, in the firstmode, the setting of the image attribute is a low frame rate, and thecontroller may output information (e.g., information for calculating aphase difference) corresponding to a phase difference between theplurality of subpixels and image data based on the first mode.

According to various embodiments of the present disclosure, in thesecond mode, the setting of the image attribute is a high frame rate,and the controller may output image data and may not output information(for example, information for calculating a phase difference)corresponding to a phase difference between the plurality of subpixelsand based on the second mode. In the third mode, the setting of theimage attribute is a higher frame rate than that of the first mode, andthe controller may output image data and information (for example,information for calculating a phase difference) corresponding to a phasedifference between the plurality of subpixels and based on the thirdmode.

According to various embodiments of the present disclosure, in thefourth mode, the setting of the image attribute is a high frame rate,and the controller may output image data generated by combining signalsof subpixels within a first unit pixel, information generated forcalculating a phase difference between pixels by using signals acquiredin subpixels within a second unit pixel, and image data generated bycombining or averaging the acquired signals.

According to various embodiments of the present disclosure, thecontroller may generate the image data by combining signals ofrespective subpixels included in the red (R) pixel and blue (B) pixelamong the plurality of unit pixels, information for calculating a phasedifference using signals of subpixels included in the green (G) pixel,and image data generated by combining or averaging the acquire signals.The controller may output the generated image and the information forcalculating the phase difference.

According to various embodiments of the present disclosure, the imagesenor may include a plurality of unit pixels, and each unit pixel mayinclude a pixel array including a plurality of subpixels, a row driverfor controlling the array of the subpixels in the unit of rows, alead-out for storing, sensing, amplifying, or outputting signals outputfrom the array of the subpixels, a time generator for providing clocksignals to the row driver and the lead-out, and a control register forstoring commands related to the operation of the image sensor.

The electronic device 101 according to an embodiment of the presentdisclosure may include the display, the image sensor 430 for acquiringan optical signal corresponding to the object 410, and the controller.The controller may identify a mode for generating an image correspondingto the object by using the optical signal, determine a setting of atleast one image attribute to be used for generating the image at leastbased on the mode, generate image data by using pixel data correspondingto the optical signal at least based on the setting, and display theimage corresponding to the object through a display functionallyconnected to the electronic device at least based on the image data.

According to various embodiments of the present disclosure, thecontroller 440 may determine a first setting of the at least one imageattribute as the setting when the mode corresponds to a first mode, andmay determine a second setting of the at least one image attribute asthe setting when the mode corresponds to a second mode.

According to various embodiments of the present disclosure, the mode mayinclude a first mode in which a plurality of subpixels included in atleast one of a plurality of unit pixels of the image senor output pixeldata of a subpixel level, respectively, a second mode in which imagedata of a unit pixel level is output through a combination of the pixeldata of the plurality of subpixels within the unit pixel, a third modein which information for calculating a phase difference between theplurality of subpixels is generated and image data of the unit pixellevel is output by combining or averaging signals of a level of theplurality of subpixels, and a fourth mode in which information forcalculating a phase difference between the plurality of subpixels isgenerated and image data of the unit pixel level is output byselectively combining or averaging pixel data of the plurality ofsubpixels.

According to various embodiments of the present disclosure, thecontroller 440 may output information corresponding to the phasedifference between the plurality of subpixels when the mode correspondsto the first mode, and may not output the information corresponding tothe phase difference between the plurality of subpixels when the modecorresponds to the second mode.

According to various embodiments of the present disclosure, thecontroller 440 may output image data and information corresponding tothe phase difference between the plurality of subpixels when the modecorresponds to the third mode.

According to various embodiments of the present disclosure, thecontroller 440 may output image data generated by combining pixel dataof subpixels within a first unit pixel, information for calculating aphase difference between pixels using pixel data acquired from subpixelswithin a second unit pixel, and image data generated by combining theacquired pixel data when the mode corresponds to the fourth mode.

The electronic device 101 according to an embodiment of the presentdisclosure may include an image sensor that acquires an optical signalcorresponding to an object, the image sensor including a plurality ofunit pixels, at least one of the plurality of unit pixels including afirst subpixel and a second pixel, and a controller functionallyconnected to the image sensor. The controller may acquire pixel datacorresponding to the object at least based on optical signal by usingthe image sensor, identifies a setting of at least one image attributeto be used for generating an image corresponding to the object,determine a phase difference of the image by using a first signalcorresponding to the first subpixel and a second signal corresponding tothe second subpixel when the setting meets predetermined conditions, andrefrain from determining the phase difference when the setting does notmeet the predetermined conditions.

According to various embodiments of the present disclosure, thecontroller 440 may generate the image of a subpixel level by using thefirst signal and the second signal when the setting meets thepredetermined conditions.

According to various embodiments of the present disclosure, thecontroller 440 may generate the image of a unit pixel level by combiningthe first signal and the second signal when the setting does not meetthe predetermined conditions.

According to various embodiments of the present disclosure, a case wherethe setting does not meet the predetermined conditions may include acase where the setting meets another predetermined condition.

According to various embodiments of the present disclosure, the at leastone image attribute may include a frame rate, and a case where thesetting meets the predetermined conditions includes a case where thesetting of the at least one image attribute corresponds to a low framerate and a case where the setting does not meet the predeterminedconditions includes a case where the setting of the at least one imageattribute corresponds to a high frame rate.

According to various embodiments of the present disclosure, the at leastone image attribute may include a size of the object, a distance betweenthe object and the image sensor, a movement speed of the object, or acombination thereof.

According to various embodiments of the present disclosure, a photodiodecorresponding to each of the first subpixel and the second subpixel maybe included.

According to various embodiments of the present disclosure, when theobject size corresponds to a predetermined range through an image dataanalysis (for example, when the object size is equal to or below 30% ofthe whole image area), the controller 440 may determine a phasedifference of the image corresponding to the object by using a firstsignal corresponding to the first subpixel and a second signalcorresponding to the second subpixel. Alternatively, when the objectsize does not correspond to the predetermined range (e.g., when theobject size is above or equal to 30%% of the whole image area), thecontroller 440 may not determine the phase difference of the imagecorresponding to the object by using a first signal corresponding to thefirst subpixel and a second signal corresponding to the second subpixel.According to an embodiment of the present disclosure, although it hasbeen described that the predetermined range is equal to or below 30%,this is only an embodiment and the predetermined range may be above 30%and may be variable.

According to various embodiments of the present disclosure, thecontroller 440 may calculate a distance between the object and the imagesensor by using at least one of the proximity sensor 240G, theillumination sensor 240K, and an infrared sensor (not shown). Thecontroller 440 may determine whether to generate information forcalculating the phase difference based on the distance between theobject and the image sensor. When the distance between the object andthe image sensor corresponds to a predetermined distance (e.g., shorterthan 5 meters), the controller 440 may determine the phase difference ofthe image corresponding to the object by using a first signalcorresponding to a first subpixel and a second signal corresponding to asecond subpixel. Alternatively, when the distance between the object andthe image sensor does not correspond to the predetermined distance (forexample, not shorter than 5 meters), the controller 440 may notdetermine the phase difference of the image corresponding to the objectby using the first signal corresponding to the first subpixel and thesecond signal corresponding to the second subpixel.

According to various embodiments of the present disclosure, thecontroller 440 may analyze a movement speed of the object through animage data analysis. For example, when the movement speed of the objectdetermined through the controller 440 corresponds to a predeterminedspeed (for example, faster than or equal to 4 km/h), the controller 440may determine the phase difference of the image corresponding to theobject by using the first signal corresponding to the first subpixel andthe second signal corresponding to the second subpixel. Alternatively,when the movement speed of the object does not correspond to thepredetermined speed (for example, not faster than or equal to 4 km/h),the controller 440 may not determine the phase difference of the imagecorresponding to the object by using the first signal corresponding tothe first subpixel and the second signal corresponding to the secondsubpixel.

FIG. 7 is a flowchart illustrating an operation for generating imagedata according to an embodiment of the present disclosure.

Referring to FIG. 7, the electronic device 101 (e.g., the controller440) may identify a mode for generating image data in operation 710. Themode may be selected or determined by the user of the electronic device.For example, the mode may be identified based on a user input acquiredthrough an application (e.g., camera 376) included in the electronicdevice 101. Further, the electronic device 101 (e.g., controller 440)may determine at least one image attribute to be used for generatingimage data and a setting of the image attribute in accordance with, forexample, the mode (e.g., photographing conditions) selected by the user.For example, the electronic device 101 (e.g., controller 440) maydetermine a setting of at least one image attribute to be used forgenerating image data in accordance with photographing conditions (forexample, sports, a person having many movements (e.g., child), a personhaving little movements (e.g., selfie), landscape, continuous shooting,or video) through the camera application (e.g., camera 376). Further,the electronic device 101 (e.g., controller 440) may generate image dataaccording to the determined setting. Among the photographing conditions,the sports may be called a first mode (third mode or fourth mode), theperson having little movements (selfie) may be called a second mode, andthe person having many movements (child) may be called a third mode or afourth mode.

When the electronic device 101 (for example, controller 440) is in apreset mode (e.g., default mode or sports), the electronic device 101may operate in the first mode. In the first mode, the controller 440 mayoutput, for example, signals of subpixel levels in all subpixels. Theelectronic device 101 (for example, controller 440) may output, forexample, signal of respective subpixel levels included in each unitpixels.

When photographing conditions (for example, a person having littlemovements (selfie), continuous shooting, or video) are selected, theelectronic device 101 (e.g., controller 440) may operate in the secondmode. In the second mode, the controller 440 may combine and read thesignals of the respective subpixel levels within each unit pixel in thesubpixels, and then output only image data of the unit pixel level. Inthis case, since the second mode has a number of pieces of data that issmaller than that in the first mode, image data may be output at ahigher speed compared to the first mode.

When the user selects photographing conditions (e.g., sports or childhaving many movements) that require high speed image processing andphase difference information, the electronic device 101 (e.g.,controller 440) may operate in the third mode. In the third mode, theimage sensor 430 may acquire signals in respective subpixels, generateinformation for calculating a phase difference between the subpixels byusing the acquired signals, and combine or average the signals acquiredin the respective subpixels at the subpixel levels, so as to outputimage data of the unit pixel level. Further, the electronic device 101may output the information for calculating the phase difference and theimage data.

When the user selects photographing conditions (e.g., sports, video, orchild having many movements) that require high speed image processingand some of the pieces of phase difference information, the electronicdevice 101 may operate in the fourth mode. In the fourth mode, thecontroller 440 may combine the signals acquired in the respectivesubpixels within a first unit pixel at subpixel levels and output imagedata of the unit pixel level. The electronic device 101 may acquiresignals in respective subpixels within a second unit pixel, generateinformation for calculating a phase difference between the subpixelswithin the second unit pixel by using the acquired signals, and combineor average the acquired signals, so as to output image data of the unitpixel level. In the fourth mode, the image data output by the first unitpixel, the image data output by the second unit pixel, and theinformation for calculating the phase difference may be output. Thefirst unit pixel may be at least one of an R pixel and a B pixel, andthe second unit pixel may be a G pixel.

The electronic device 101 may control the operation of the image sensor430 in order to output image data in the identified mode. The mode mayinclude a first mode in which signals of the subpixel level are outputfrom all subpixels included in the image sensor, a second mode in whichimage data of the unit pixel level is output by combining signals ofrespective subpixels within the unit pixel, a third mode in whichsignals of the pixel level are acquired from respective subpixels withinthe unit pixel, information for calculating a phase difference betweensubpixels is generated through the acquired signals, and image data ofthe unit pixel level is output by combining or averaging signals of thesubpixel level acquired from the respective subpixels within the unitpixel at the subpixel level, and a fourth mode in which image data ofthe unit pixel level is output by combining signals of respectivesubpixels within a first unit pixel (e.g., R pixel or B pixel) at thesubpixel level, information for calculating a phase difference usingsignals acquired from respective subpixels within a second unit pixel(e.g., G pixel) is output, and image data of the unit pixel level isoutput by combining or averaging the acquired signals at the sub pixellevel.

In operation 712, the electronic device 101 may identify whether imagedata is generated. The electronic device 101 may identify whether anoptical signal corresponding to an object is received through the imagesensor 430. The electronic device 101 may identify a mode for generatingimage data through the received optical signal corresponding to theobject through the image sensor 430.

In operation 714, when the mode is determined as the first mode, theelectronic device 101 may acquire signals (or pixel data) fromrespective subpixels in operation 716. When the image data is output inthe first mode, the electronic device 101 may control the image sensorto output image data in the first mode. The electronic device 101 maydetect signals output by respective subpixels among a plurality ofsubpixels included in the unit pixel of the pixel array 431 of the imagesensor 430. The electronic device 101 may process signals output inrespective sub pixels included in the same unit pixel.

In operation 718, the electronic device 101 may output signals ofrespective subpixel levels within each unit pixel. The electronic device101 may output signals of respective subpixel levels in each unit pixelof the pixel array 431.

When the mode is determined as the second mode in operation 724, theelectronic device 101 may combine and read signals of respectivesubpixels within the unit pixel at subpixel levels in operation 726. Inoperation 728, the electronic device 101 may output only image data ofthe unit pixel level through the added signal at the sub pixel levels.The electronic device 101 may control the image sensor to output imagedata in the second mode. The electronic device 101 may combine and readsignals of subpixels within the unit pixel of the pixel array 431 of theimage sensor 430 at subpixel level and output image data of the unitpixel level.

The electronic device 101 may output image data at a high frame rate(e.g., 60 frames per second) in the second mode, and may not generate oroutput information for calculating a phase difference between pixelswith respect to the image data output by the second mode. The secondmode may be applied to a scenario (e.g., continuous shooting) in whichonly image data is acquired at a high speed. In the second mode, summingof the subpixels may include pixel levels (e.g., analog domain), so thatnoise may be small.

When the mode is determined as the third mode in operation 732, theelectronic device 101 may acquire signals from respective subpixels inoperation 734. Further, the electronic device 101 may control the imagesensor 430 to output image data in the third mode. In operation 736, theelectronic device 101 may generate, for example, information forcalculating a phase difference between subpixels included in the unitpixel of the pixel array 431 of the image sensor 430.

In operation 738, the electronic device 101 may generate, for example,image data of the unit pixel level through the signals acquired in thesubpixels within the unit pixel. The electronic device 101 may controlthe image sensor 430 to output image data in the third mode. Theelectronic device 101 may generate image data at a high frame rate(e.g., 60 frames per second) in the third mode that is faster than thefirst mode.

In operation 740, the electronic device 101 may output the generatedimage data and information for calculating a phase difference betweensubpixels within the unit pixel. The electronic device 101 may outputthe generated image data and the information for calculating the phasedifference together.

When the mode is determined as the fourth mode in operation 744, theelectronic device 101 may selectively combine the signals of respectivesubpixels according to each unit pixel in operation 746. When the imagedata is output in the fourth mode, the electronic device 101 may controlthe image sensor 430 to output image data in the fourth mode. In thefourth mode, the electronic device 101 may combine and output thesignals of the subpixels at subpixel levels only with respect to aparticular unit pixel and output signal of pixel levels of respectivesub pixels with respect to other (or the remaining) unit pixels. Theelectronic device 101 may selectively combine signals of subpixelswithin the unit pixel of the pixel array 431 of the image sensor 430.For example, in the fourth mode, image data of an R pixel or B pixellevel is output by combining signals of subpixels including R pixels orB pixels, and information for calculating a phase difference betweensubpixels using signals acquired in respective subpixels including Gpixels and image data of a G pixel level through combining or averagingthe signals acquired in the G pixels may be output. The G pixel may havea weighted value larger than that of the R pixel and the B pixel. Forexample, human eyes have the highest sensitivity to green. For thisreason, the weighted value of the G pixel may be 0.7, the weighted valueof the R pixel may be 0.2, and the weighted value of the B pixel may be0.1. Accordingly, in the fourth mode, image data of the unit pixel levelmay be output by combining the signals of the subpixels including the Rpixels and the B pixels at the subpixel levels and signals of respectivesubpixels levels may be output with respect to the G pixels.

In operation 748, the electronic device 101 may generate information forcalculating a phase difference between subpixels within the unit pixel.For example, the electronic device 101 may output information forcalculating a phase difference between pixels by using the signalsacquired in the subpixels within the G pixel. The information mayinclude filtered data for calculating the phase difference.

In operation 750, the electronic device 101 may generate image data ofthe unit pixel level. For example, the electronic device 101 may gatherimage data generated in the R pixel, image data generated in the Gpixel, and image data generated in the B pixel. In operation 752, theelectronic device 101 may output the gathered image data andinformation. The electronic device 101 (e.g., image sensor 430) maygenerate the image data at a high frame rate (e.g., 60 frames persecond) in the fourth mode faster than the first mode.

In the fourth mode, the generated data and the information forcalculating the phase difference may be output individually orsimultaneously. The fourth mode may be applied to a scenario (e.g.,continuous shooting or sports) that simultaneously uses, for example,high speed image processing (e.g., 60 frames per second) and subpixelinformation (e.g., information for calculating the phase difference).The fourth mode may have a smaller amount of data that should be outputthrough an interface line, for example, compared to the first mode.Alternatively, the fourth mode may operate with low power (e.g., 100%reduction compared to the first mode) by reducing a clock (e.g., 60%reduction compared to the first mode) and reducing a pixel resolution(e.g., ½ resolution compared to the first mode), and may increase aframe rate (e.g., double speed with respect to a maximum interface speedcompared to the first mode) into a maximum speed.

According to various embodiments, the electronic device 101 may outputin real time image data based on a changed mode in accordance withgeneration of the mode change while the image data is output through thedisplay 160.

FIG. 8A illustrates an example in which the unit pixel of the imagesensor includes two subpixels according to an embodiment of the presentdisclosure, and FIG. 8B illustrates an example in which the unit pixelof the image sensor includes four subpixels according to an embodimentof the present disclosure.

Referring to FIGS. 8A and 8B, the image sensor 430, according to anembodiment of the present disclosure, may include a plurality of unitpixels, and each unit pixel may include a plurality of subpixels. Thepixel marked by R may perform an operation for acquiring a pixel imagefor a red color, the pixel marked by G may perform an operation foracquiring a pixel image for a green color, and the pixel marked by B mayperform an operation for acquiring a pixel image for a blue color.

Human eyes have the highest sensitivity to the green color, so the Gpixel may mainly use two filters.

One unit pixel may include two or more subpixels (e.g., photodiodes).Each unit pixel may include at least one micro lens. Further, each unitpixel may include at least one color filter. The color filter may belocated between the micro lens and the subpixel. The subpixel mayreceive light having a visible ray of at least some area having passedthrough the micro lens and the color filter, and may output the light asdata. Accordingly, the unit pixel may output one sub pixel level signaland include a plurality of pieces of data in one pixel.

The unit pixel may combine signals of two or more subpixel levelsincluded in the unit pixel, and may output the added signal as one pieceof data. Further, the unit pixel may output phase difference informationfor calculating a phase difference of light incident to the included twoor more subpixels. The information may include filtered data forcalculating the phase difference. For example, in a case of the pixelincluding two subpixels as illustrated in FIG. 8A, the information mayinclude information for calculating a phase difference between left andright pixels. Further, when the pixel includes four subpixels asillustrated in FIG. 8B, the information may include information forcalculating a phase difference between top and bottom pixels by usingtop and bottom subpixels and information for calculating a phasedifference between left and right pixels by using left and rightsubpixels. The information may include information for calculating aphase difference by using subpixels located diagonally. The informationmay output only information for calculating a phase difference betweenpixels of a particular color. For example, only in the G pixel thatreceives a relatively large amount of light among RGB pixels, the colorinformation and the information for calculating the phase difference areoutput. In the remaining R pixel and B pixel, only the color informationmay be output.

Although the Bayer pattern based on the red color, the green color, andthe blue color is illustrated for the pixel array, the presentdisclosure is not limited thereto and various filter patterns may beused.

Each unit pixel may include an optical detection element and a filter.The pixel R may include a filter which allows a red light and/or aninfrared light to pass therethrough, the pixel G may include a greenlight filter, and the pixel B may include a blue light filter. Each unitpixel may penetrate, for example, not only the red light, the greenlight, or the blue light but also the infrared light, so that detectioninformation generated by the light having passed through each pixel mayinclude noise by the infrared light. When noise by the infrared light issmall in the detection information, color information may be acquired,for example, based on the detection information. Unlike the above, whenit is required to remove noise by the infrared light, detectioninformation from which the noise by the infrared light is removed may begenerated through a proper processing procedure and color informationmay be acquired based on the generated detection information.

FIG. 9 illustrates an example of a process for outputting image data ina first mode according to an embodiment of the present disclosure.

Referring to FIG. 9, the controller 440 may output signals of subpixellevels included in a plurality of unit pixels of the image sensor 430,and may generate or output data. The controller 440 may output, forexample, a signal of a subpixel level from each of a plurality ofsubpixels included in each unit pixel. A pixel array 901 of the imagesensor 430 may include a plurality of unit pixels 902, 903, 904, and905, and each unit pixel may include a plurality of subpixels. Forexample, each unit pixel may include two photodiodes. Alternatively,each unit pixel may include four photodiodes. For example, a first unitpixel 902 may include two subpixels. The first unit pixel 902 may be theR pixel, a second unit pixel 903 may be the G pixel, a third unit pixel904 may be the G pixel, and a fourth unit pixel 905 may be the B pixel.

The R pixel 902 may include two subpixels (e.g., photodiodes), and mayoutput a signal 911 of a first subpixel level and/or a signal 912 of asecond subpixel level. Further, the G pixel 903 may include twosubpixels (e.g., photodiodes), and may output a signal 913 of a thirdsubpixel level and a signal 914 of a fourth subpixel level.

The G pixel 904 may include two subpixels (e.g., photodiodes), and mayoutput a signal 931 of a fifth subpixel level and/or a signal 932 of asixth subpixel level. Further, the B pixel 905 may include two subpixels(e.g., photodiodes), and may output a signal 933 of a seventh subpixellevel and/or a signal 934 of an eighth subpixel level.

FIG. 10 illustrates an example of a process for outputting image data ina second mode according to an embodiment of the present disclosure.

Referring to FIG. 10, the electronic device 101 may output only imagedata of the unit pixel level by combining signals of subpixels withinthe unit pixel of the image sensor 430. In this case, a number of piecesof data becomes smaller compared to the case of FIG. 9, so that only theimage data may be output at a high speed. The pixel array of the imagesensor 430 may include a plurality of unit pixels, and each unit pixelmay include a plurality of subpixels. For example, each unit pixel mayinclude two photodiodes. Alternatively, each unit pixel may include fourphotodiodes. For example, each of a first unit pixel to a fourth unitpixel 1002, 1003, 1004, and 1005 may include two subpixels. The firstunit pixel 1002 may be the R pixel, the second unit pixel 1003 may bethe G pixel, the third unit pixel 1004 may be the G pixel, and thefourth unit pixel 1005 may be the B pixel.

The R pixel 1002 may include two subpixels (e.g., photodiodes), and mayoutput image data 1011 of the unit pixel level by combining signals inrespective subpixels. The G pixel 1003 may include two subpixels (e.g.,photodiodes), and output image data 1012 of the unit pixel level bycombining signals of respective subpixels. Further, each unit pixelaccording to the line may combine and output signals of respective subpixels. The G pixel 1004 may include two subpixels (e.g., photodiodes)and output image data 1021 of the unit pixel level by combining data ofrespective subpixels, and the B pixels 1005 may include two subpixels(e.g., photodiodes) and output image data 1022 of the unit pixel levelby combining data of respective subpixels.

The electronic device 101 may generate image data of the unit pixellevel by combining signals of the plurality of subpixels. Further, theelectronic device 101 may also output image data of the unit pixel levelgenerated in each unit pixel. The electronic device 101 may generateimage data by combining light amounts by respective subpixels includedin the R pixel 1002, and may generate image data 1012 by combining lightamounts by respective subpixels included in the G pixel 1003. Further,the electronic device 101 may combine the image data 1011 and 1012. Inaddition, the electronic device 101 may generate image data 1021 bycombining data by respective subpixels included in the G pixel 1004, andmay generate image data 1022 by combining data by respective subpixelsincluded in the B pixel 1005.

The second mode may be applied to a scenario (for example, continuousshooting) for acquiring only image data at a high speed. In this case,since summing of subpixels includes unit pixels, noise is small and thushigh definition image data may be acquired. Further, the electronicdevice 101 may not generate or output, for example, information forcalculating a phase difference between subpixels within the unit pixel.

FIG. 11A illustrates an example of a process for outputting image datain a third mode according to an embodiment of the present disclosure.

FIG. 11B illustrates an example of a process for outputting image datain a fourth mode according to an embodiment of the present disclosure.

Referring to FIG. 11A, the electronic device 101 (e.g., image sensor430) may acquire signals 1121 a, 1121 b, 1122 a, 1122 b, 1123 a, 1123 b,1124 a, and 1124 b from subpixels 1111 a, 111 b, 1112 a, 1112 b, 1113 a,1113 b, 1114 a, and 1114 b of unit pixels 1111, 1112, 1113, and 1114.The electronic device 101 may generate information 1130 for calculatinga phase difference by using the acquired signals 1121 a, 1121 b, 1122 a,1122 b, 1123 a, 1123 b, 1124 a, and 1124 b. The electronic device 101may generate image data of the unit pixel level by combining oraveraging signals acquired from the plurality of subpixels of each unitpixel. The electronic device 101 may generate first image data 1125 ofthe unit pixel level by combining or averaging the signals 1121 a and1121 b acquired from the subpixels 1111 a and 1111 b of the first unitpixel 1111, generate second image data 1126 of the unit pixel level bycombining or averaging the signals 1122 a and 1122 b acquired from thesubpixels 1112 a and 1112 b of the second unit pixel 1112, generatethird image data 1127 of the unit pixel level by combining or averagingthe signals 1123 a and 1123 b acquired from the subpixels 1113 a and1113 b of the third unit pixel 1113, and generate fourth image data 1128of the unit pixel level by combining or averaging the signals 1124 a and1124 b acquired from the subpixels 1114 a and 1114 b of the fourth unitpixel 1114. Further, the electronic device 101 may output the generatedimage data 1125, 1126, 1127, and 1128 and the information 1130. Forexample, the generated image data 1125, 1126, 1127, and 1128 and theinformation 1130 may be output individually or simultaneously.

The electronic device 101 (e.g., controller 440) may acquire signals1151 a, 1151 b, 1152 a, 1152 b, 1153 a, 1153 b, 1154 a, and 1154 b fromsubpixels 1141 a, 141 b, 1142 a, 1142 b, 1143 a, 1143 b, 1144 a, and1144 b of unit pixels 1141, 1142, 1143, and 1144. The electronic device101 may generate information 1160 for calculating a phase difference byusing the acquired signals 1151 a, 1151 b, 1152 a, 1152 b, 1153 a, 1153b, 1154 a, and 1154 b. The electronic device 101 may generate image dataof the unit pixel level by combining or averaging signals acquired fromthe plurality of subpixels of each unit pixel. The electronic device 101may generate fifth image data 1155 of the unit pixel level by combiningor averaging the signals 1151 a and 1151 b acquired from the subpixels1141 a and 1141 b of the fifth unit pixel 1141, generate sixth imagedata 1156 of the unit pixel level by combining or averaging the signals1152 a and 1152 b acquired from the subpixels 1142 a and 1142 b of thesixth unit pixel 1142, generate seventh image data 1157 of the unitpixel level by combining or averaging the signals 1153 a and 1153 bacquired from the subpixels 1143 a and 1143 b of the seventh unit pixel1143, and generate eighth image data 1158 of the unit pixel level bycombining or averaging the signals 1154 a and 1154 b acquired from thesubpixels 1144 a and 1144 b of the eighth unit pixel 1144. Further, theelectronic device 101 may output the generated image data 1155, 1156,1157, and 1158 and the information 1160. For example, the generatedimage data 1155, 1156, 1157, and 1158 and the information 1160 may beoutput individually or simultaneously. The generated first image data toeighth image data 1125, 1126, 1127, 1128, 1155, 1156, 1157, and 1158 andthe information 1130 and 1160 may be output together.

The pixel array 431 of the image sensor 430 may include a plurality ofunit pixels, and each unit pixel may include a plurality of subpixels.For example, each unit pixel may include two photodiodes. Alternatively,each unit pixel may include four photodiodes. The controller 440 maycombine or average signals of subpixels within the unit pixel includedin the image sensor 430. The controller 440 may simultaneously outputthe combined data or averaged data and information for calculating aphase difference between the subpixels within each unit pixel. Further,the first unit pixel 1111 may include the R pixel, and the second unitpixel 1112 may include the G pixel. The third unit pixel 1113 mayinclude the R pixel, and the fourth unit pixel 1114 may include the Gpixel. The fifth unit pixel 1141 may include the G pixel, and the sixthunit pixel 1142 may include the B pixel. The seventh unit pixel 1143 mayinclude the G pixel, and the eighth unit pixel 1144 may include the Bpixel.

The electronic device 101 (e.g., image sensor 430) may generateinformation for calculating a phase difference between subpixels of eachunit pixel.

Referring to FIG. 11B, the electronic device 101 (e.g., image sensor430) may output image data 1175 of the unit pixel level by combiningsignals of subpixels 1161 a and 1161 b of a first unit pixel 1161 at thesubpixel level as indicated by reference numeral 1171, and mayindividually output signals 1172 a and 1172 b acquired from subpixels1162 a and 1162 b of a second unit pixel 1162 without combining thesignals. The electronic device 101 (e.g., controller 440) may generateinformation 1179 for calculating a phase difference by using the signals1172 a and 1172 b acquired from the second unit pixel 1162 and outputimage data 1176 of the unit pixel level by combining or averaging theacquired signals 1172 a and 1172 b. The electronic device 101 may outputimage data 1177 of the unit pixel level by combining signals ofsubpixels 1163 a and 1163 b of a third unit pixel 1163 at the subpixellevel as indicated by reference numeral 1173, and may individuallyoutput signals 1174 a and 1174 b acquired from subpixels 1164 a and 1164b of a fourth unit pixel 1164 without combining the signals. Theelectronic device 101 may generate information 1179 for calculating aphase difference by using the signals 1174 a and 1174 b acquired fromthe fourth unit pixel 1164 and output image data 1178 of the unit pixellevel by combining or averaging the acquired signals 1174 a and 1174 b.Further, the electronic device 101 may output the generated image data1175, 1176, 1177, and 1178 and the information 1179. For example, thegenerated image data 1175, 1176, 1177, and 1178 and the information 1179may be output individually or simultaneously.

The electronic device 101 may individually output signals 1191 a and1191 b acquired from subpixels 1181 a and 1181 b of a fifth unit pixel1181 without combining the signals, and may output image data 1192 ofthe unit pixel level by combining signals of subpixels 1182 a and 1182 bof a sixth unit pixel 1182 at the subpixel level. The electronic device101 may generate, for example, information 1199 for calculating a phasedifference by using the signals 1191 a and 1191 b acquired from thefifth unit pixel 1181 and output image data 1194 of the unit pixel levelby combining or averaging the acquired signals 1191 a and 1191 b.Further, the electronic device 101 may individually output signals 1193a and 1193 b of subpixels 1183 a and 1183 b of a seventh unit pixel 1183without combining the signals, and output image data 1197 of the unitpixel level by combining signals of subpixels 1184 a and 1184 b of aneighth unit pixel 1184 at the subpixel level. The electronic device 101may generate, for example, information 1199 for calculating a phasedifference by using the signals 1193 a and 1193 b acquired from theseventh unit pixel 1183 and output image data 1194 of the unit pixellevel by combining or averaging the acquired signals 1191 a and 1191 b.Further, the electronic device 101 may output, for example, thegenerated image data 1195, 1196, 1197, and 1198 and the information1199. For example, the generated image data 1194, 1195, 1196, and 1197and the information 1199 may be output individually or simultaneously.According to various embodiments, the 1175, 1176, 1177, 1178, 1195,1196, 1197, and 1198 of the unit pixel level generated in respectiveunit pixels may be combined and output, and the information 1179 and1199 for calculating the phase difference between the subpixels withinthe second unit pixel and between the subpixels within the fourth unitpixel may be added to the combined image data and transmitted.

The first unit pixel 1161 may include the R pixel, and the second unitpixel 1162 may include the G pixel. The third unit pixel 1163 mayinclude the R pixel, and the fourth unit pixel 1164 may include the Gpixel. The fifth unit pixel 1181 may include the G pixel, and the sixthunit pixel 1182 may include the B pixel. The seventh unit pixel 1183 mayinclude the G pixel, and the eighth unit pixel 1184 may include the Bpixel. As described above, in the fourth mode, the image data generatedby selectively combining the signals of the subpixels within the firstunit pixel (or the third unit pixel, the sixth unit pixel, or the eighthunit pixel), the information generated for calculating the phasedifference between the pixels using the signals acquired from thesubpixels within the second unit pixel (or the fourth unit pixel, thefifth unit pixel, or the seventh unit pixel), and the image datagenerated by combining or averaging the acquired signals may be combinedand output.

According to various embodiments, the electronic device 101 (forexample, image sensor 430) may generate information for calculating aphase difference between subpixels (for example, between subpixels 1162a and 1162 b, between subpixels 1164 a and 1164 b, between subpixels1181 a and 1181 b, and between subpixels 1183 a and 1183 b) of the Gpixels 1162, 1164, 1184, and 1183. The information may be informationfor calculating the phase difference between the subpixels and mayinclude filtered data.

The fourth mode, according to various embodiments, may include an outputmode (e.g., sports or video) having a high frame rate (60 frame persecond) and a phase difference between subpixels. The fourth mode may beapplied to a scenario (e.g., continuous shooting or sports) using highspeed image processing and information for calculating the phasedifference between subpixels. For example, when AF is required during ahigh speed preview (e.g., high speed image processing or 60 frames persecond), the fourth mode is executed. The fourth mode may have a smalleramount of data which should be output through an interface line comparedto the first mode, the fourth mode may be executed with low power (e.g.,10% reduction compared to the first mode) by reducing a clock (e.g., 60%reduction compared to the first mode) and reducing a pixel resolution(e.g., ½ resolution compared to the first mode). Further, the fourthmode may perform an output at a maximum speed supported by theinterface, thereby increasing a frame rate. In such a fourth mode, forexample, data output from some subpixels may be combined and output, andthus an output speed may increase compared to the first mode.

FIG. 12 illustrates an example of a process for selectively combiningsignals of the subpixel level in the fourth mode according to anembodiment of the present disclosure.

Referring to FIG. 12, the electronic device 101 (e.g., image sensor 430)may combine signals acquired through two subpixels of the unit pixel(e.g., R pixel) at the subpixel level as indicated by reference numeral1212, charge the signal in a capacitor of the corresponding photodiode,and perform a lead-out output. Further, the electronic device 101 (e.g.,controller 440) may charge signals 1220 and 1230 acquired thoughrespective subpixels of the two subpixels of the unit pixel (forexample, G pixel) in respective capacitors of the correspondingphotodiode and perform a lead-out output. In FIG. 12, in order tofurther improve the output speed of the fourth mode, image data of theunit pixel level may be output by selectively combining signals of somepixels, so that the image data may be generated faster than the thirdmode and simultaneously a phase difference between subpixels within theG pixel may be output.

FIG. 13A illustrates an example of a channel (e.g., a mobile industryprocessor interface (MIPI) or an MIPI virtual channel (VC) fortransmitting the generated image data and/or the information forcalculating the phase difference according to an embodiment of thepresent disclosure.

FIG. 13B illustrates an example for transmitting image data andinformation for calculating a phase difference through a predeterminedchannel (e.g., MIPI or MIPI VC) according to an embodiment of thepresent disclosure.

Referring to FIGS. 13A and 13B, the electronic device (e.g., controller440) may transmit information 1312 and 1313 for calculating a phasedifference that is generated using four lines 1311 for transmittingimage data and all unit pixels related to the four lines 1311 fortransmitting the image data.

FIG. 13C illustrates an example for transmitting image data andinformation for calculating a phase difference through a predeterminedchannel (e.g., MIPI or MIPI VC) according to an embodiment of thepresent disclosure.

Referring to FIG. 13C, the electronic device (e.g., controller 440) maytransmit information 1322 for calculating a phase difference that isgenerated using four lines 1321 for transmitting image data and someunit pixels related to the four lines 1321 for transmitting the imagedata. The four lines 1311 and 1321 for transmitting the image data andlines for transmitting the information 1312, 1313, and 1322 forcalculating the phase difference may be different. The four lines 1311and 1321 for transmitting the image data may use, for example, the MIPI,and the information 1312, 1313, and 1322 for calculating the phasedifference may use the MIPI VC.

Referring to FIGS. 13A to 13C, the image data generated according to anembodiment of the present disclosure may be transmitted through a firstline to a fourth line 1301, 1302, 1303, and 1304, and the informationmay be transmitted through a separate channel 1305. Alternatively, theimage data generated according to an embodiment of the presentdisclosure may be transmitted through at least one of the first line tothe fourth line 1301, 1302, 1303, and 1304.

Referring to FIG. 13B, the electronic device (e.g., controller 440) maytransmit image data 1311 and information 1312 and 1313 for calculatingthe phase difference generated using all unit pixels related to theimage data 1311. The image data 1311 and the information 1312 and 1313for calculating the phase difference may be transmitted through onechannel or different channels. When the image sensor 430 does notprovide a high dynamic range (HDR), information 1312 and 1313 forcalculating a phase difference may be generated using all unit pixelsrelated to the image data 1311 as illustrated in FIG. 13B. The imagedata 1311 may include the first to fourth lines 1301, 1302, 1303, and1304 illustrated in FIG. 13A. Further, the information 1312 and 1313 forcalculating the phase difference may include PAF data illustrated inFIG. 13A.

Referring to FIG. 13C, the electronic device (e.g., controller 440) maytransmit image data 1321 and information 1322 for calculating a phasedifference generated using some of all unit pixels related to the imagedata 1321. The image data 1321 and the information 1322 for calculatingthe phase difference may be transmitted through one channel or differentchannels. When the image sensor 430 provides a HDR, the controller 440may generate information 1322 for calculating a phase difference byusing some of all unit pixels (e.g., pixels supporting long exposure)related to the image data 1321 as illustrated in FIG. 13C. The imagedata 1321 may include the first to fourth lines 1301, 1302, 1303, and1304 illustrated in FIG. 13A. Further, the information 1322 forcalculating the phase difference may include PAF data illustrated inFIG. 13A.

According to various embodiments of the present disclosure, thecontroller 440 may efficiently generate phase difference information(e.g., information calculated using information for calculating thephase difference) by simultaneously or sequentially transmitting imagedata and information for calculating the phase difference. For example,the controller 440 may prevent the problem of omitting the phasedifference information or delaying a time when the phase differenceinformation is generated by simultaneously or sequentially transmittingimage data and information for calculating the phase difference. Thecontroller 440 may prevent the problem (generation of a time differenceof 1 frame) generated when the information for calculating the phasedifference is transmitted after all the image data is transmitted bysimultaneously or sequentially transmitting the four lines 1311 and 1321of the image data and the information 1312, 1313, and 1322 forcalculating the phase difference.

FIG. 14 is a flowchart illustrating an operation for generating an imageaccording to an embodiment of the present disclosure.

Referring to FIG. 14, the electronic device 101 (e.g., image sensor 430)may acquire, for example, an optical signal corresponding to an objectin operation 1410. The optical signal may include various informationrequired for generating image data by using signals acquired fromrespective subpixels included in the image sensor 430 of the electronicdevice 101. The electronic device 101 (e.g., controller 440) maygenerate information for calculating a phase difference betweensubpixels through the signals acquired from the subpixels and (or)generate image data. The image sensor 430 may include a plurality ofunit pixels, and each unit pixel may include one of the R pixel, the Gpixel, and the B pixel.

The electronic device 101 may identify a setting of at least one imageattribute in operation 1412. For example, the electronic device 101 mayidentify the image attribute of an image to be output through thedisplay 160. The image attribute may include an object size, a distancebetween the object and the image sensor, a movement speed of the object,a frame rate of the image to be output, or a combination thereof. Theelectronic device 101 may identify the image attribute to determine ornot determine the phase difference through the signals corresponding tothe respective subpixels.

The electronic device 101 may determine whether the setting meetspredetermined conditions in operation 1414. When the setting meets thepredetermined conditions the electronic device 101 may control or set atleast one element of the electronic device 101 to generate image data byusing a first signal of a first subpixel and a second signal of a secondsubpixel.

The electronic device 101 (e.g., image sensor 430) may determine a phasedifference corresponding to the object by using the first signal of thefirst subpixel and the second signal of the second subpixel in operation1416. The electronic device 101 (e.g., controller 440) may generateimage data by using the first signal and the second signal in operation1418. The electronic device 101 may generate image data of the unitpixel level by using the first signal and the second signal. Forexample, the electronic device 101 may generate the image data bycombining the first signal and the second signal or using an averagevalue of the first signal and the second signal.

When the setting does not meet the predetermined conditions in operation1414 (e.g., when the setting is within a second predetermined range),the electronic device 101 may not determine, for example, the phasedifference corresponding to the object by using the first signal of thefirst subpixel and the second signal of the second subpixel in operation1420. Further, in operation 1422, the electronic device 101 may generateimage data by combining the first signal and the second signal. Theelectronic device 101 may generate image data of the unit pixel level bycombining, for example, the first signal and the second signal. A casewhere the setting does not meet the predetermined conditions may includea case where the setting meets other predetermined conditions.

FIG. 15 is a flowchart illustrating an operation for generating imagedata according to an embodiment of the present disclosure.

Referring to FIG. 15, the operation for generating the image data mayinclude an operation for identifying a mode for generating image data,an operation for determining a setting of image attributes, an operationfor generating the image data based on the setting, and an operation foroutputting the generated image data.

Referring to FIG. 15, the electronic device 101 (e.g., controller 440)may identify the mode for generating image data in operation 1510. Theelectronic device 101 may determine the mode for generating the imagedata that matches conditions selected by the user. Alternatively, theelectronic device 101 may selectively execute the mode according to auser's situation or attributes of a photographed image. The attributesof the image may vary depending on sports, person, landscape, or video,and the electronic device 101 may selectively operate in accordance withthe mode according to the attributes.

The electronic device 101 (e.g., controller 440) may identify whetherthe mode for generating the image data corresponds to a first mode inwhich signals of a plurality of subpixels included in at least one unitpixel among a plurality of unit pixels are individually output. Theelectronic device 101 may identify a second mode in which signals of aplurality of subpixels within each unit pixel are combined at thesubpixel level and image data of the unit pixel level is output. Theelectronic device 101 may identify a third mode in which information forcalculating a phase difference is generated through signals acquiredfrom a plurality of subpixels within each unit pixel and image data ofthe unit pixel level is generated by combining or averaging the acquiredsignals. The electronic device 101 may identify a fourth mode in whichsignals of respective subpixels within a first unit pixel (e.g., R pixelor B pixel) are combined at the subpixel level and image data of theunit pixel level is output, information for calculating a phasedifference is output by using signals acquired from respective subpixelswithin a second unit pixel (e.g., G pixel), and image data of the unitpixel level is output by combining or averaging the acquired signals.The first mode corresponds to a mode selected when high speed AF (e.g.,AF using phase difference information) is needed, and the second modecorresponds to a mode selected when a high frame rate (60 frames persecond) is needed. The third mode and the fourth mode correspond tomodes selected when the high speed AF (e.g., AF using phase differenceinformation) and the high frame rate (e.g., 60 frames per second) areneeded. The fourth mode corresponds to a mode in which image data may begenerated at a higher frame rate compared to the first mode and imagedata of each unit pixel level and information for calculating a phasedifference between subpixels within the G pixel may be generated. In thefourth mode, image data of the unit pixel level may be generated throughselective combination of signals of the subpixels and the informationmay be added to the generated image data and output. In the fourth mode,image data of the unit pixel level may be output through a combinationof signals of subpixels included in the R pixel at the subpixel level,information for calculating a phase difference may be output by usingsignals acquired from respective subpixels included in the G pixel,image data of the unit pixel level may be generated by combining oraveraging the acquired signals, and image data of the unit pixel levelmay be output through a combination of signals of respective subpixelsincluded in the B pixel at the subpixel level. Further, in the fourthmode, the generated image data may be gathered and the information maybe added to the gathered data, and then the image data and theinformation may be output together.

In operation 1512, the electronic device 101 may determine a setting ofimage attributes. The electronic device 101 may determine a setting ofat least one image attribute to be used for generating image data atleast based on the mode identified in operation 1512.

In operation 1514, the electronic device 101 may generate image databased on the setting. When the setting corresponds to the first mode,the electronic device 101 may provide, for example, high speed AF (e.g.,AF using phase difference information) (e.g., 60 frames per second).Further, the electronic device 101 may generate image data at a lowframe rate (e.g., 30 frames per second). According to an embodiment ofthe present disclosure, although it has been described that the lowframe rate corresponds to 30 frame per second or 15 frames per second,this is only an embodiment and the low frame rate according to thepresent disclosure may include a lower or higher frame rate than 30frame per second or 15 frames per second described above according tothe specification of the electronic device 101 or technologydevelopment.

When the identified setting corresponds to the second mode, theelectronic device 101 may generate the image at a frame rate (e.g., 60frames per second) that is faster than the frame rate of the first modeand may not generate information for calculating a phase differencebetween subpixels within the unit pixel. Further, since signals acquiredfrom respective subpixels within the unit pixel may be output in thefirst mode and an image of the unit pixel level may be generated andoutput in the second mode through a combination of signals acquired fromrespective subpixels within the unit pixel at the subpixel level, thesecond mode may have a smaller amount of data compared to the firstmode, so that only the image data may be output at a high speed throughthe second mode. According to an embodiment of the present disclosure,although it has been described that the high frame rate corresponds to60 frames per second, this is only an embodiment and the high frame rateaccording to the present disclosure may include a lower or higher framerate than 60 frame per second described above according to thespecification of the electronic device 101 or technology development.

When the identified setting corresponds to the third mode, theelectronic device 101 may generate information for calculating a phasedifference between subpixels within each unit pixel by using signalsacquired from the subpixels within each unit pixel and may generateimage data of the unit pixel level by combining or averaging the signalsof the subpixels within each unit pixel at the subpixel level. Forexample, since the amount of processed data (or data amount) becomessmaller in the third mode compared to the first mode, the electronicdevice 101 may output the image data and the information for calculatingthe phase difference at a higher speed compared to the first mode.

When the identified setting corresponds to the fourth mode, a dataamount output through the interface line may be smaller than that of thefirst mode. Alternatively, the fourth mode may be executed with lowpower (e.g., 10% reduction compared to the first mode) by reducing aclock (e.g., 60% reduction compared to the first mode) and reducing apixel resolution (e.g., ½ resolution compared to the first mode), andmay increase the frame rate into a maximum rate. The frame rate (e.g.,double rate compared to the first mode with respect to a maximuminterface rate) corresponding to the fourth mode is a rate forprocessing frames at a relatively high speed, and may include 60 framesper second. Although 60 frames per second has been described in anembodiment of the present disclosure, this is only an embodiment and thepresent disclosure may include a lower or higher frame rate than 60frames per second according to the specification of the electronicdevice 101 or technology development.

In operation 1516, the electronic device 101 may output the generatedimage data. The electronic device 101 may combine and output, forexample, the image data generated at each unit level in operation 1514.Further, the electronic device 101 may process, for example, theplurality of combined image data through a post-processing process anddisplay the image data through the display 160 functionally connected tothe electronic device. When the mode or the setting changes while theimage data is displayed through the display 160, the electronic device101 may generate image data by using at least one of the frame rate andthe resolution corresponding to the changed setting and display thegenerated image data in real time.

The operations (e.g., operations 710 to 752, operations 1410 to 1422, oroperations 1510 to 1516) described in the processes and methodsillustrated in FIGS. 7, 14, and 15 may be performed in a sequential,parallel, repetitive, or heuristic type. For example, the operations maybe performed in a different order, some of the operations may beomitted, or other operations may be added.

According to various embodiments of the present disclosure, a method, byan electronic device including an image sensor that acquires an opticalsignal corresponding to an object and a controller that controls theimage sensor, may include an operation of identifying a mode forgenerating an image corresponding to the object by using the opticalsignal, an operation of determining a setting of at least one imageattribute to be used for generating the image at least based on themode, an operation of generating image data by using pixel datacorresponding to the optical signal at least based on the setting, andan operation of displaying the image corresponding to the object througha display that is functionally connected to the electronic device atleast based on the image data.

According to various embodiments of the present disclosure, theoperation of determining the setting may include an operation ofdetermining a first setting of the at least one image attribute as thesetting when the mode corresponds to a first mode, and an operation ofdetermining a second setting of the at least one image attribute as thesetting when the mode corresponds to a second mode.

According to various embodiments of the present disclosure, the mode mayinclude a first mode in which a plurality of subpixels included in atleast one of a plurality of unit pixels of the image senor output pixeldata of a subpixel level, respectively, a second mode in which imagedata of a unit pixel level is output through a combination of the pixeldata of the plurality of subpixels within the unit pixel, a third modein which information for calculating a phase difference between theplurality of subpixels is generated and image data of the unit pixellevel is output by combining or averaging signals of a level of theplurality of subpixels, and a fourth mode in which information forcalculating a phase difference between the plurality of subpixels isgenerated and image data of the unit pixel level is output byselectively combining or averaging pixel data of the plurality ofsubpixels.

According to various embodiments of the present disclosure, informationcorresponding to the phase difference between the plurality of subpixelsmay be output based on the first mode.

According to various embodiments of the present disclosure, image datamay be output and information corresponding to the phase differencebetween the plurality of subpixels may not be output based on the secondmode.

According to various embodiments of the present disclosure, image dataand information corresponding to the phase difference between theplurality of subpixels may be output based on the third mode.

According to various embodiments of the present disclosure, image datagenerated by combining pixel data of subpixels within a first unitpixel, information for calculating or averaging a phase differencebetween pixels using pixel data acquired from subpixels within a secondunit pixel, and image data generated by combining or averaging theacquired pixel data may be output based on the fourth mode.

According to various embodiments of the present disclosure, the firstunit pixel may include a Red (R) pixel or Blue (B) pixel, and the secondunit pixel may include a Green (G) pixel.

According to various embodiments of the present disclosure, theoperation of generating the image data may include an operation ofacquiring pixel data corresponding to an object at least based on anoptical signal, an operation of identifying a setting of at least oneimage attribute to be used for generating an image corresponding to theobject, an operation of, when the setting meets predeterminedconditions, determining a phase difference of the image by using a firstsignal corresponding to the first subpixel and a second signalcorresponding to the second subpixel, and an operation of, when thesetting does not meet the predetermined conditions, refraining fromdetermining the phase difference.

According to various embodiments of the present disclosure, when thesetting meets the predetermined conditions, an operation of generatingthe image of a subpixel level by using the first signal and the secondsignal may be included. For example, when the setting meets thepredetermined conditions, the controller 440 may generate an image of asubpixel level of each of the first signal and the second signal.

According to various embodiments of the present disclosure, when thesetting does not meet the predetermined conditions, an operation ofgenerating the image of a unit pixel level by combining the first signaland the second signal may be included.

According to various embodiments of the present disclosure, a case wherethe setting may not meet the predetermined conditions may include a casewhere the setting meets another predetermined condition.

According to various embodiments of the present disclosure, the at leastone image attribute may include a frame rate, and a case where thesetting meets the predetermined conditions may include a case where thesetting of the at least one image attribute corresponds to a low framerate and a case where the setting does not meet the predeterminedconditions may include a case where the setting of the at least oneimage attribute corresponds to a high frame rate.

According to various embodiments of the present disclosure, the at leastone image attribute may include a size of the object, a distance betweenthe object and the image sensor, a movement speed of the object, or acombination thereof.

According to various embodiments of the present disclosure, a photodiodecorresponding to each of the first subpixel and the second subpixel maybe included.

According to various embodiments of the present disclosure, theoperation of generating the image data may include an operation ofacquiring an optical signal corresponding to an object through an imagesensor, the image sensor including a plurality of unit pixels, at leastone of the plurality of unit pixels including a first subpixel and asecond pixel, an operation of acquiring pixel data corresponding to theobject at least based on an optical signal by using the image sensor, anoperation of identifying a setting of at least one image attribute to beused for generating an image corresponding to the object, an operationof determining a phase difference of the image by using a first signalcorresponding to the first subpixel and a second signal corresponding tothe second subpixel when the setting meets predetermined conditions, andan operation of refraining from determining the phase difference whenthe setting does not meet the predetermined conditions.

According to various embodiments of the present disclosure, when thesetting meets the predetermined conditions, an operation of generatingthe image of a subpixel level by using the first signal and the secondsignal may be included.

According to various embodiments of the present disclosure, when thesetting does not meet the predetermined conditions, an operation ofgenerating the image of a unit pixel level by combining the first signaland the second signal may be included.

According to various embodiments of the present disclosure, a case wherethe setting may not meet the predetermined conditions may include a casewhere the setting meets another predetermined condition.

According to various embodiments of the present disclosure, the at leastone image attribute may include a frame rate, and a case where thesetting meets the predetermined conditions may include a case where thesetting of the at least one image attribute corresponds to a low framerate and a case where the setting does not meet the predeterminedconditions may include a case where the setting of the at least oneimage attribute corresponds to a high frame rate.

According to various embodiments of the present disclosure, the at leastone image attribute may include a size of the object, a distance betweenthe object and the image sensor, a movement speed of the object, or acombination thereof.

According to various embodiments of the present disclosure, photodiodescorresponding to the first subpixel and the second subpixel may beincluded.

The term “module” as used herein may, for example, mean a unit includingone of hardware, software, and firmware or a combination of two or moreof them. The “module” may be interchangeably used with, for example, theterm “unit”, “logic”, “logical block”, “component”, or “circuit”. The“module” may be a minimum unit of an integrated component element or apart thereof. The “module” may be a minimum unit for performing one ormore functions or a part thereof. The “module” may be mechanically orelectronically implemented. For example, the “module” according to thepresent disclosure may include at least one of an application-specificintegrated circuit (ASIC) chip, a field-programmable gate arrays (FPGA),and a programmable-logic device for performing operations which has beenknown or are to be developed hereinafter.

According to various embodiments of the present disclosure, at leastsome of the devices (e.g., modules or functions thereof) or the method(e.g., operations) according to the present disclosure may beimplemented by a command stored in a computer-readable storage medium ina programming module form. The instruction, when executed by a processor(e.g., the processor 120), may cause the one or more processors toexecute the function corresponding to the instruction. Thecomputer-readable storage medium may be, for example, the memory 130.

The computer readable recoding medium may include a hard disk, a floppydisk, magnetic media (e.g., a magnetic tape), optical media (e.g., acompact disc read only memory (CD-ROM) and a DVD), magneto-optical media(e.g., a floptical disk), a hardware device (e.g., a read only memory(ROM), a random access memory (RAM), a flash memory), and the like. Inaddition, the program instructions may include high class languagecodes, which can be executed in a computer by using an interpreter, aswell as machine codes made by a compiler. The aforementioned hardwaredevice may be configured to operate as one or more software modules inorder to perform the operation of the present disclosure, and viceversa.

According to various embodiments of the present disclosure, a storagemedium having instructions stored therein is provided. The instructionsare configured to allow one or more processors to perform one or moreoperations when being executed by the one or more processors. The one ormore operations, by an electronic device including an image sensor thatacquires an optical signal corresponding to an object and a controllerthat controls the image sensor, an operation of identifying a mode forgenerating an image corresponding to the object by using the opticalsignal, an operation of determining a setting of at least one imageattribute to be used for generating the image at least based on themode, an operation of generating image data by using pixel datacorresponding to the optical signal at least based on the setting, andan operation of displaying the image corresponding to the object througha display functionally connected to the electronic device at least basedon the image data.

According to various embodiments of the present disclosure, a storagemedium having instructions stored therein is provided. The instructionsare configured to allow one or more processors to perform one or moreoperations when being executed by the one or more processors. The one ormore operations may include an operation of acquiring pixel datacorresponding to an object at least based on an optical signal by usingan image sensor including a plurality of unit pixels, at least one unitpixel including a first subpixel and a second subpixel for acquiring theoptical signal corresponding to the object, an operation of identifyinga setting of at least one image attribute to be used for generating animage corresponding to the object, an operation of determining a phasedifference of the image by using a first signal corresponding to thefirst subpixel and a second signal corresponding to the second subpixelwhen the setting meets predetermined conditions, and an operation ofrefraining from determining the phase difference when the setting doesnot meet the predetermined conditions.

The programming module according to the present disclosure may includeone or more of the aforementioned components or may further includeother additional components, or some of the aforementioned componentsmay be omitted. Operations executed by a module, a programming module,or other component elements according to various embodiments of thepresent disclosure may be executed sequentially, in parallel,repeatedly, or in a heuristic manner. Furthermore, some operations maybe executed in a different order or may be omitted, or other operationsmay be added.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. A portable communication device comprising: amemory; a processor; a display; and a camera comprising: an image sensorcomprising a plurality of unit pixels including a first unit pixel and asecond unit pixel, the first unit pixel comprising a first subpixel anda second subpixel, and the second unit pixel comprising a third subpixeland a fourth subpixel, and a controller electrically connected to theprocessor via an interface, the controller being configured to: obtain afirst unit pixel signal and a second unit pixel signal from the firstunit pixel and the second unit pixel, respectively, the first unit pixelsignal comprising a first subpixel signal corresponding to the firstsubpixel and a second subpixel signal corresponding to the secondsubpixel, and the second unit pixel signal comprising a third subpixelsignal corresponding to the third subpixel and a fourth subpixel signalcorresponding to the fourth subpixel; generate first phase differenceinformation corresponding to the first unit pixel and second phasedifference information corresponding to the second unit pixel, the firstphase difference information being generated based at least in part on afirst difference between the first subpixel signal and the secondsubpixel signal, the second phase difference information being generatedbased at least in part on a second difference between the third subpixelsignal and the fourth subpixel signal; generate first image datacorresponding to the first unit pixel based at least in part on thefirst unit pixel signal, generate second image data corresponding to thesecond unit pixel based at least in part on the second unit pixelsignal; transmit, to the processor via the interface, the first phasedifference information and the first image data with respect to thefirst unit pixel; and transmit, to the processor via the interface, thesecond phase difference information and the second image data withrespect to the second unit pixel.
 2. The portable communication deviceof claim 1, wherein the controller is further configured to, based onthe image sensor being set in a specific mode, refrain from thegenerating of the first and second phase difference information.
 3. Theportable communication device of claim 2, wherein the controller isfurther configured to, as at least part of the generating of the firstimage data and the second image data, combine, at a subpixel level, thefirst subpixel signal and the second subpixel signal corresponding to arespective unit pixel.
 4. The portable communication device of claim 1,wherein the controller is further configured to, as at least part of thegenerating of the first image data and the second image data, combine,at a unit pixel level, the first subpixel signal and the second subpixelsignal corresponding to a respective unit pixel.
 5. The portablecommunication device of claim 1, wherein the controller is furtherconfigured to, as at least part of the generating of the first imagedata and the second image data, combine the first subpixel signal andthe second subpixel signal corresponding to a respective unit pixel of aportion of a plurality of unit pixels at a subpixel level, and combinethe first subpixel signal and the second subpixel signal correspondingto a respective unit pixel of another portion of the plurality of unitpixels at a unit pixel level.
 6. The portable communication device ofclaim 1, wherein the interface comprises a mobile industry processorinterface (MIPI) including a first transmission channel and a secondtransmission channel, wherein the first image data and the second imagedata are transmitted via the first transmission channel, and wherein thefirst phase difference information and the second phase differenceinformation are transmitted via the second transmission channel.
 7. Theportable communication device of claim 1, wherein the first unit pixelcorresponds to at least one of a red color channel or a blue colorchannel, wherein the first unit pixel is configured to accumulate anaccumulated signal from the first subpixel and the second subpixel ofthe first unit pixel, and wherein the first image data is generatedusing the accumulated signal read out from the first unit pixel.
 8. Theportable communication device of claim 1, wherein the second unit pixelcorresponds to a green color channel, and wherein the second unit pixelis configured to generate a first green subpixel signal corresponding tothe first subpixel signal and a second green subpixel signalcorresponding to the second subpixel signal.
 9. The portablecommunication device of claim 1, wherein the controller is furtherconfigured to: identify a mode for the generating of the first imagedata and the second image data; based on the mode, determine a type ofphase difference calculation from among a plurality of types of phasedifference calculations; and based at least in part on the type of phasedifference calculation, selectively perform the generating of the firstphase difference information and the second phase differenceinformation.
 10. The portable communication device of claim 1, whereinthe first unit pixel signal is generated based on a first exposure timeand the second unit pixel signal is generated based on a second exposuretime longer than the first exposure time.
 11. A method for operating anelectronic device, the method comprising: obtaining, from an imagesensor by a controller included in a camera of the electronic device, afirst unit pixel signal and a second unit pixel signal from a first unitpixel and a second unit pixel, respectively, of the image sensor, theimage sensor comprising a plurality of unit pixels including the firstunit pixel and the second unit pixel, the first unit pixel comprising afirst subpixel and a second subpixel, the second unit pixel comprising athird subpixel and a fourth subpixel, the first unit pixel signalcomprising a first subpixel signal corresponding to the first subpixeland a second subpixel signal corresponding to the second subpixel, thesecond unit pixel signal comprising a third subpixel signalcorresponding to the third subpixel and a fourth subpixel signalcorresponding to the fourth subpixel; generating, by the controller, foreach unit pixel of the plurality of unit pixels, first phase differenceinformation corresponding to the first unit pixel and second phasedifference information corresponding to the second unit pixel, the firstphase difference information being generated based at least in part on afirst difference between the first subpixel signal and the secondsubpixel signal, and the second phase difference information beinggenerated based at least in part on a second difference between thethird subpixel signal and the fourth subpixel signal; generating, by thecontroller, first image data corresponding to the first unit pixel basedat least in part on the first unit pixel signal, generating, by thecontroller, second image data corresponding to the second unit pixelbased at least in part on the second unit pixel signal; transmitting, toa processor via an interface, the first phase difference information andthe first image data with respect to the first unit pixel; andtransmitting, to the processor via the interface, the second phasedifference information and the second image data with respect to thesecond unit pixel.
 12. The method of claim 11, further comprising: basedon the image sensor being set in a specific mode, refraining from thegenerating of the first and second phase difference information.
 13. Themethod of claim 11, further comprising: as at least part of thegenerating of the first image data and the second image data, combining,at a subpixel level, the first subpixel signal and the second subpixelsignal corresponding to a respective unit pixel.
 14. The method of claim11, further comprising: as at least part of the generating of the firstimage data and the second image data, combining, at a unit pixel level,the first subpixel signal and the second subpixel signal correspondingto a respective unit pixel.
 15. The method of claim 11, furthercomprising: as at least part of the generating of the first image dataand the second image data, combining the first subpixel signal and thesecond subpixel signal corresponding to a respective unit pixel of aportion of a plurality of unit pixels at a subpixel level, and combiningthe first subpixel signal and the second subpixel signal correspondingto a respective unit pixel of another portion of the plurality of unitpixels at a unit pixel level.
 16. The method of claim 11, wherein theinterface comprises a mobile industry processor interface (MIPI)including a first transmission channel and a second transmissionchannel, wherein the first image data and the second image data aretransmitted via the first transmission channel, and wherein the firstphase difference information and the second phase difference informationare transmitted via the second transmission channel.
 17. The method ofclaim 11, wherein the first unit pixel corresponds to at least one of ared color channel or a blue color channel, wherein the first unit pixelis configured to accumulate an accumulated signal from the firstsubpixel and the second subpixel of the first unit pixel, and whereinthe first image data is generated using the accumulated signal read outfrom the first unit pixel.
 18. The method of claim 11, wherein theobtaining comprises: wherein the second unit pixel corresponds to agreen color channel, and wherein the second unit pixel is configured togenerate a first green subpixel signal corresponding to the firstsubpixel signal and a second green subpixel signal corresponding to thesecond subpixel signal.
 19. The method of claim 11, further comprising:identifying a mode for the generating of the first image data and thesecond image data; based on the mode, determining a type of phasedifference calculation from among a plurality of types of phasedifference calculations; and based at least in part on the type of phasedifference calculation, selectively performing the generating of thefirst phase difference information and the second phase differenceinformation.
 20. The method of claim 11, wherein the first unit pixelsignal is generated based on a first exposure time and the second unitpixel signal is generated based on a second exposure time longer thanthe first exposure time.